Implement create_pretrained_project() for Pytorch engine#2897
Merged
maximpavliv merged 14 commits intopytorch_dlcfrom Feb 25, 2025
Merged
Implement create_pretrained_project() for Pytorch engine#2897maximpavliv merged 14 commits intopytorch_dlcfrom
maximpavliv merged 14 commits intopytorch_dlcfrom
Conversation
create_pretrained_project() for Pytorch engine
create_pretrained_project() for Pytorch engine
Contributor
Author
|
The Tensorflow implementation of |
ae721ac to
ea78c81
Compare
9324d57 to
48f0fd4
Compare
n-poulsen
approved these changes
Feb 25, 2025
Contributor
n-poulsen
left a comment
There was a problem hiding this comment.
LGTM @maximpavliv, thanks! just one minor comment
48f0fd4 to
0c4db27
Compare
In original/tf methods (f.ex. analyze_videos), default value of videotype was "", this default value was not treated properly in list_videos_in_folder()
Move read_config_as_dict, write_config and pretty_print to core
When calling read_config, the superanimal yaml config in the repo was modified
0c4db27 to
a67974b
Compare
This was referenced Feb 25, 2025
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This Pull Request adds Pytorch support for the
create_pretrained_project()method (and closes #2732)Specifically:
create_pretrained_project()becomes acompat.py-like method, and calls the specific method implementation based on the specifiedengineparameter.The old
create_pretrained_project()is renamed tocreate_pretrained_project_tensorflow()and a newcreate_pretrained_project_pytorch()method with the Pytorch-shuffle logic is added.The existing usages of
create_pretrained_project()are also adapted.create_pretrained_project_*implementations call_create_training_datasets_metadata().This method has been cherry-picked from [WIP] DLC 3.0 - Review documentation #2881(where a first version,
_create_and_save_fake_metadata(), has been implemented as a temporary fix), and adapted to work with both Tensorflow and Pytorch. It creates themetadata.yamlfile in the training-datasets directory, and fills it with a first shuffle corresponding to the downloaded modelzoo model._process_videos(), and called by bothcreate_pretrained_project_*implementations.individualsparameter is added tocreate_new_project()and tocreate_pretrained_project(), so that the list of individuals can be automatically filled in the different configuration files of the new project.create_pretrained_project(engine=Engine.PYTORCH)is added.This Pull Request also does some fixes to issues that have been spotted during its development:
list_videos_in_folder(): in original (Tensorflow-implementation) methods (f.ex.analyze_videos()),videotypeargument's value by default was""(instead ofNone), and this default argument value was not treated properly.get_super_animal_project_cfg()was usingread_config()to read the superanimal project configuration template embedded into the DeepLabCut repository, which then modified the file in the DeepLabCut codebase. Now, the method reads the template withread_config_as_dict(), which doesn't modify the file.read_config_as_dict(),write_config()andpretty_print()methods todeeplabcut.core.config_utils.pymodule.testscript_superanimal_adaptation.py,testscript_superanimal_inference.pyandtestscript_superanimal_transfer_learning.py) have been fixed.