add --exclude and tool.polylith.exclude

[fraser-langton]

We are using a different test structure that to the single test directory approach encouraged here, we are putting a tests dir in each module in a more package driven apprach, as a result poly check commands now complain that the test modules are not in the pyproject of projects.

I think the easiest and most generalised solution would be to allow configuration of the glob pattern poly searches to build the project space?

Ruff I think is a good example for config like this

poly info --exclude "**/tests/**"

# Multiple exclusions
poly check \
  --exclude "**/test_*" \
  --exclude "**/tests/**" \

[tool.polylith]
namespace = "ns"
git_tag_pattern = "stable-*"

# Add exclude patterns for Polylith commands
exclude = [
    "**/test_*",
    "**/tests/**",
]

[DavidVujic]

That sounds like a bug, it shouldn't complain about that. Can you share an example of the structure you use so I can test it out?

Maybe you already know about the "TDD" theme that is already supported in Polylith. It's about having source code and tests in each component. The drawback in Python is that MyPy doestn't really support it by default, and need a lot of extra config to allow it.

[DavidVujic]

[fraser-langton]

we are using loose theme and yes for the reason you mentioned we aren't using the tdd theme although that is the more pure python package approach it is not as easy to configure or navigate.

So we are just doing loose theme, but having a tests dir inside the modules, purely to keep the tests closer to the code

[DavidVujic]

That sounds like it should work out of the box. When you mention that the poly check command complains about the test modules: do you have the tests referenced somewhere in the pyproject.toml files, or import them in a component, maybe?

Otherwise it should work, as my example above.

[fraser-langton]

don't know if there is some config I am missing but here is a demo https://github.com/fraser-langton/polylith-demo

[DavidVujic]

Aha, it is complaining on the third-party dependencies. I thought you meant that it was about the tests themselves.

I think this makes sense, because the test module is part of the brick. Any project that includes the brick would need the used third-party dependencies in there. I understand that you would want to ignore those, but I think that could be risky and open up for missing dependencies.

Instead, I would recommend to add the used libraries as optional dependencies in the projects.

Example:

[project.optional-dependencies]
dev = [
  "pytest",
  "requests",
]

Additionally, there's a workaround (but I wouldn't recommend it because it looks weird).
The check command has support for an --alias option, where you can tell the poly tool to treat one dependency as another. In this case you could do something like this:

# not recommended, not the intent of the alias option -  but works
uv run poly check --alias <a-library-you-use-and-is-there-for-sure>=requests

[fraser-langton]

I have ways around it, I am suggesting that adding the exclude option would be a good addition to make the tool more generalised.

I think the use of dev dependencies in what is supposed to be a production only project is fraught, here is a scenario:

• I am using httpx in tests, poly check says I should add it so I add to dev
• later down the track httpx is added as an actual dep in a brick
• the httpx is still a dev dep and poly check says nothing
• deployment is done and httpx is not in production artifact

I would point out that the tach check-external is an example of a highly generalised command (and has exclude, as such I have applied it in a few times in different scenarios, in the scenario I describe it would actually be better than poly check

Perhaps one solution would be another theme, but I think that is too prescriptive and the option to simply exclude/include through toml and cli args is more powerful and follows conventions of other popular python tools.

[DavidVujic]

I have ways around it, I am suggesting that adding the exclude option would be a good addition to make the tool more generalised.

I think the use of dev dependencies in what is supposed to be a production only project is fraught, here is a scenario:

* I am using `httpx` in tests, `poly check` says I should add it so I add to dev

* later down the track `httpx` is added as an actual dep in a brick

* the `httpx` is still a dev dep and `poly check` says nothing

* deployment is done and `httpx` is not in production artifact

I would point out that the tach check-external is an example of a highly generalised command (and has exclude, as such I have applied it in a few times in different scenarios, in the scenario I describe it would actually be better than poly check

Perhaps one solution would be another theme, but I think that is too prescriptive and the option to simply exclude/include through toml and cli args is more powerful and follows conventions of other popular python tools.

I'll give it some thought!

I don't like the ignore pattern, because I think it could be too broad and open up for other problems. I have some ideas of other types of ignore patterns that I will try out.

The way you have it set up: that means also that your tests will be part of the deployments (because they are part of the actual brick). Maybe that's okay, I just want to highlight that.

[DavidVujic]

@fraser-langton I have released a new version, that I believe will solve this problem. The tool already respects any project-specific exclude pattern when building wheels. With the new release, the poly check and poly libs commands also takes any exclusions into consideration.

I hope you will find this useful!

Example usage

In a project-specific toml-file, when using the hatchling build-backend:

[tool.hatch.build.targets.wheel]
exclude = ["**/the_thing_to_exclude/*"]

Thank you for sharing your feedback and details about how you use Polylith! Thanks to you, the tool has become better ⭐ 👏

You'll find the implementation details and more build-backend examples in #387

[DavidVujic]

I'll close this one. Please reopen or create a new issue if you find bugs or something that doesn't work as expected.

[fraser-langton]

Sorry @DavidVujic missed the reply to this, but works great! Thanks!