Skip to content

Add a windows "known-paths" locator. - #14675

Merged
Eric Snow (ericsnowcurrently) merged 37 commits into
microsoft:mainfrom
ericsnowcurrently:pyenvs-component-windows-known-paths-locator
Dec 10, 2020
Merged

Add a windows "known-paths" locator.#14675
Eric Snow (ericsnowcurrently) merged 37 commits into
microsoft:mainfrom
ericsnowcurrently:pyenvs-component-windows-known-paths-locator

Conversation

@ericsnowcurrently

@ericsnowcurrently Eric Snow (ericsnowcurrently) commented Nov 9, 2020

Copy link
Copy Markdown

Note that all but ~100 lines of this PR is tests.

  • Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR).
  • Title summarizes what is changing.
  • [ ] Has a news entry file (remember to thank yourself!).
  • Appropriate comments and documentation strings in the code.
  • Has sufficient logging.
  • [ ] Has telemetry for enhancements.
  • Unit tests & system/integration tests are added/updated.
  • [ ] Test plan is updated as appropriate.
  • [ ] package-lock.json has been regenerated by running npm install (if dependencies have changed).
  • [ ] The wiki is updated with any design decisions/details.

@codecov-io

Codecov (codecov-io) commented Nov 10, 2020

Copy link
Copy Markdown

Codecov Report

Merging #14675 (6faa56b) into main (3698950) will decrease coverage by 0%.
The diff coverage is 63%.

@@           Coverage Diff           @@
##            main   #14675    +/-   ##
=======================================
- Coverage     65%      65%    -1%     
=======================================
  Files        552      552            
  Lines      25926    26026   +100     
  Branches    3678     3693    +15     
=======================================
+ Hits       16918    16926     +8     
- Misses      8315     8413    +98     
+ Partials     693      687     -6     
Impacted Files Coverage Δ
src/client/activation/jedi/languageServerProxy.ts 25% <0%> (-2%) ⬇️
src/client/activation/node/analysisOptions.ts 55% <0%> (-16%) ⬇️
src/client/common/utils/exec.ts 20% <0%> (-2%) ⬇️
src/client/pythonEnvironments/base/envsCache.ts 68% <0%> (-2%) ⬇️
...s/discovery/locators/services/KnownPathsService.ts 48% <0%> (ø)
src/client/activation/node/languageServerProxy.ts 25% <15%> (-2%) ⬇️
src/client/common/utils/text.ts 50% <28%> (-19%) ⬇️
...t/activation/languageServer/languageServerProxy.ts 90% <75%> (-1%) ⬇️
.../locators/services/virtualEnvironmentIdentifier.ts 92% <75%> (+<1%) ⬆️
.../pythonEnvironments/common/externalDependencies.ts 63% <87%> (-10%) ⬇️
... and 12 more

@ericsnowcurrently
Eric Snow (ericsnowcurrently) force-pushed the pyenvs-component-windows-known-paths-locator branch from 292d176 to c79f3cd Compare November 10, 2020 18:16
@sonarqubecloud

Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 0 Security Hotspots to review)
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@karthiknadig

Copy link
Copy Markdown
Member

You need to rebase. The API in index has changed a bit.

@karrtikr Kartik Raj (karrtikr) left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems we have a disagreement in the way testing is supposed to be done, I am concerned this will make testing the component as a whole harder.

Comment thread src/client/pythonEnvironments/base/locators/lowLevel/windowsKnownPathsLocator.ts Outdated
Comment thread src/client/pythonEnvironments/base/locators/lowLevel/windowsKnownPathsLocator.ts Outdated
Comment thread src/test/utils/fs.ts
@ericsnowcurrently
Eric Snow (ericsnowcurrently) marked this pull request as draft November 17, 2020 18:08
Comment thread src/test/utils/text.ts Outdated
Comment thread src/test/utils/fs.ts
Comment thread src/test/testing/helper.ts Outdated
@ericsnowcurrently
Eric Snow (ericsnowcurrently) force-pushed the pyenvs-component-windows-known-paths-locator branch from c79f3cd to ce255b1 Compare December 1, 2020 22:44
@ericsnowcurrently
Eric Snow (ericsnowcurrently) marked this pull request as ready for review December 1, 2020 22:46
@ericsnowcurrently Eric Snow (ericsnowcurrently) changed the title Add a windows known-paths locator. Add a windows "known-paths" locator. Dec 1, 2020
Comment thread src/client/common/utils/exec.ts
Comment thread src/client/pythonEnvironments/base/locators/lowLevel/windowsKnownPathsLocator.ts Outdated
async function* generator() {
try {
for await (const executable of findInterpretersInDir(entry)) {
const okay = await isValidAndExecutable(executable);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should only be returning python.exe or python to avoid returning duplicates (as done in other locators using isStandardPythonBinary).

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what the reducer is supposed to do. Each low-level locator is responsible only for tracking any valid environment within its knowledge domain. They should be focused on the one job and leave any other responsibilities to composite locators.

@karrtikr Kartik Raj (karrtikr) Dec 8, 2020

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reducer doesn't resolve symlinks. cc Karthik Nadig (@karthiknadig)
We decided to delegate the responsibility to the locators as they know best regarding that. (for eg. windows store locators symlinks for the same env maybe located in different folders, so there's no way for the reducer to resolve that, hence the locator itself makes sure to not return symlinks).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kartik Raj (@karrtikr) is right. Depending on the context, which the individual locator have, we may or may not want to resolve symlinks.

@ericsnowcurrently Eric Snow (ericsnowcurrently) Dec 10, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been addressed.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I should be checking for symlinks? How does that work on Windows?

@ericsnowcurrently Eric Snow (ericsnowcurrently) Dec 10, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I guess I'm still not clear on why ReducingLocator wouldn't take care of this. Is it just that we want to favor python.exe when it is in the same directory as python3.8.exe? Why wouldn't we do that in the reducer (via a call to some helper)?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can address the question of a better approach separately. I'll use isStandardPythonBinary() for now.

@karrtikr Kartik Raj (karrtikr) Dec 10, 2020

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it just that we want to favor python.exe when it is in the same directory as python3.8.exe? Why wouldn't we do that in the reducer (via a call to some helper)?

We don't want to do it for every locator, for eg. in Windows store we prioritize python3.8.exe over python.exe, hence logic is specific to the locator and not the reducer.

Comment thread src/client/pythonEnvironments/base/locators/lowLevel/windowsKnownPathsLocator.ts Outdated
Comment thread src/client/pythonEnvironments/common/commonUtils.ts Outdated

constructor() {
const dirLocators: DirFilesLocator[] = getSearchPathEntries()
.map((dirname) => new DirFilesLocator(dirname, PythonEnvKind.Unknown));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be marked as PythonEnvKind.Global instead.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be System, OtherGlobal, Custom, Venv, or really any of them. We simply do not know. Hence Unknown.

@karrtikr Kartik Raj (karrtikr) Dec 10, 2020

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's Venv it should be resolved in the reducer as it'll also be discovered by Venv locators. If we're not able to distinguish between System, OtherGlobal, Custom for now, we should just go with System IMO. Having an Unknown type could mean anything.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note we also had an OKR to move away from Unknown type.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We currently also detect in as Unknown, so nevermind.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This locator should not be responsible for guessing/determining the environment's kind. That is the responsibility of kind-specific locators. So ultimately every env that this locator identifies should also be identified by at least one other locator that can determine the kind. If we end up keeping this locator in the long term then it will only be as a fallback when we are missing the appropriate locator. Regardless, rather than a dedicated $PATH-based locator we should have a $PATH helper that the kind-specific locators use. If we kept a generic $PATH-based locator then it would just be a light wrapper around that helper.

(FWIW, all the same things apply to any generic Windows registry locator, and probably a couple of others.)

All that said, we have diverged a bit from the kind-specific focus of locators and this PR is not the place to fix that. 😄

Comment thread src/client/pythonEnvironments/base/locators/lowLevel/windowsKnownPathsLocator.ts Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look good to me. Approving assuming Kartik's comments will be addressed.

@karrtikr Kartik Raj (karrtikr) left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once the filter comment is addressed.

@ericsnowcurrently
Eric Snow (ericsnowcurrently) force-pushed the pyenvs-component-windows-known-paths-locator branch from b87cc62 to 68151a3 Compare December 10, 2020 23:40
@ericsnowcurrently
Eric Snow (ericsnowcurrently) merged commit df2dff3 into microsoft:main Dec 10, 2020
@ericsnowcurrently
Eric Snow (ericsnowcurrently) deleted the pyenvs-component-windows-known-paths-locator branch December 10, 2020 23:55
Karthik Nadig (karthiknadig) pushed a commit to karthiknadig/vscode-python that referenced this pull request Dec 11, 2020
Karthik Nadig (karthiknadig) added a commit to karthiknadig/vscode-python that referenced this pull request Dec 14, 2020
Kartik Raj (karrtikr) pushed a commit to karrtikr/vscode-python that referenced this pull request Dec 14, 2020
Kartik Raj (karrtikr) pushed a commit to karrtikr/vscode-python that referenced this pull request Dec 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog No news entry required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants