Skip to content

Update TensorFlow backend for Python 3.12 compatibility (Keras 2 via tf-keras) - LTS DLC 2#3091

Merged
MMathisLab merged 18 commits intoLTS-DLC-2from
maxim/relax_dependencies_lts_dlc2
Nov 4, 2025
Merged

Update TensorFlow backend for Python 3.12 compatibility (Keras 2 via tf-keras) - LTS DLC 2#3091
MMathisLab merged 18 commits intoLTS-DLC-2from
maxim/relax_dependencies_lts_dlc2

Conversation

@maximpavliv
Copy link
Contributor

@maximpavliv maximpavliv commented Sep 9, 2025

Description

This PR backports the TensorFlow backend fixes from main branch PR to the DLC 2 LTS branch, in order to ensure compatibility with Python 3.12 (needed due to Google Colab’s migration from Python 3.11 → 3.12).

The motivation and general solution are the same as described in the main branch PR:

  • Unpin TensorFlow (installing ≥2.16).
  • Force switch to Keras 2 by installing tf-keras and setting the TF_USE_LEGACY_KERAS=1 environment variable.
  • Add a _tf_legacy.py shim to declare the environment variables and restore compatibility with tf_keras.legacy_tf_layers.
  • Work around TensorFlow/wrapt bug (WRAPT_DISABLE_EXTENSIONS=1).
  • Relax other pinned dependencies (tables, PySide6, etc.) for Python 3.12.
  • Extend CI/CD to test against Python 3.11 and 3.12.

See the main branch PR for full details.


LTS-specific changes

In addition to the above, the following changes were required specifically for the LTS branch:

  • CI/CD triggers: Fixed workflow triggers (were missing in LTS branch).
  • TensorFlow import order: Moved import tensorflow out of deeplabcut/__init__.py and into deeplabcut/pose_estimation_tensorflow/__init__.py (as already done in main branch), to ensure _tf_legacy.py runs before TensorFlow is imported.
  • Python 3.12 and random.samle(): Converted Graph edges to list before calling random.sample (dict views are no longer supported in 3.12). This fix matches main.
  • Functional tests: Removed MobileNets from functional tests (not available anymore, also already removed in main).
  • Dependency updates: Synced with main branch updates (intel-openmp, matplotlib, torch).
  • CircleCI: Updated Docker image to python:3.12 (since latest TensorFlow does not provide wheels for 3.8 anymore).

Testing

  • Verified unit and functional tests run successfully in CI/CD for Python 3.11 and 3.12 on Linux, Windows, and macOS.
  • Verified Colab notebooks with TensorFlow backend run with the simplified installation.
  • Same caveat as in main: MacOS with tensorflow-metal should be manually tested.

@maximpavliv maximpavliv changed the base branch from main to LTS-DLC-2 September 9, 2025 12:06
@maximpavliv maximpavliv added the WORK IN PROGRESS! developers are currently working on this feature... stay tuned. label Sep 9, 2025
@maximpavliv maximpavliv force-pushed the maxim/relax_dependencies_lts_dlc2 branch from 5f90898 to 504b556 Compare September 9, 2025 13:20
@maximpavliv maximpavliv force-pushed the maxim/relax_dependencies_lts_dlc2 branch from 864dce3 to de31770 Compare September 9, 2025 20:24
@maximpavliv maximpavliv added Installation COLAB Tensorflow 2 dependencies Pull requests that update a dependency file and removed WORK IN PROGRESS! developers are currently working on this feature... stay tuned. labels Sep 10, 2025
@maximpavliv maximpavliv changed the title [WIP] Fix dependencies for installation - LTS DLC 2 Fix dependencies for installation - LTS DLC 2 Sep 10, 2025
@maximpavliv maximpavliv changed the title Fix dependencies for installation - LTS DLC 2 Update TensorFlow backend for Python 3.12 compatibility (Keras 2 via tf-keras) - LTS DLC 2 Sep 10, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR updates DeepLabCut's dependencies to support newer versions of TensorFlow, PyTorch, and Python, while simplifying COLAB installation instructions. The main changes transition from pinned legacy versions to more flexible version constraints that support modern ML frameworks.

  • Removes version pins on core dependencies (tables, pyside6) and updates TensorFlow/PyTorch to latest versions
  • Adds tf-keras as explicit dependency to support TensorFlow 2.13+
  • Simplifies COLAB notebook installation from multi-step CUDA workarounds to single pip install command

Reviewed Changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
setup.py Removes version pins on tables/pyside6, adds tf-keras dependency, removes TensorFlow upper version constraint for non-Windows platforms
requirements.txt Updates PyTorch to 2.7.1, TensorFlow to >=2.0, adds tf-keras, adds platform-specific intel-openmp, adjusts matplotlib version
examples/testscript_multianimal.py Removes mobilenet_v2_0.35 from test models list
examples/testscript.py Removes mobilenet_v2_0.35 from test models list
examples/COLAB/*.ipynb Simplifies installation from multi-step CUDA 11.8 workaround to single deeplabcut[tf] install
deeplabcut/utils/auxfun_multianimal.py Fixes NetworkX 3.x compatibility by converting EdgeView to list
deeplabcut/pose_estimation_tensorflow/_tf_legacy.py Adds new module to configure TensorFlow legacy Keras mode and fix tf-keras import paths
deeplabcut/pose_estimation_tensorflow/init.py Imports _tf_legacy module and moves TensorFlow logging suppression from main init.py
deeplabcut/init.py Removes TensorFlow import and logging suppression (moved to pose_estimation_tensorflow)
.github/workflows/python-package.yml Updates CI to test Python 3.10-3.12 on LTS-DLC-2 branch with macos-latest
.github/workflows/codespell.yml Updates branch target to LTS-DLC-2
.circleci/config.yml Updates CircleCI Python image from 3.8 to 3.12
Comments suppressed due to low confidence (1)

setup.py:42

  • The torch dependency in setup.py is unpinned while requirements.txt specifies torch==2.7.1 (a non-existent version). These should be consistent. Consider using the same version constraint in both files, such as torch>=2.0 or a valid pinned version.
        "torch",

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@MMathisLab MMathisLab merged commit 9455c05 into LTS-DLC-2 Nov 4, 2025
17 checks passed
@MMathisLab MMathisLab deleted the maxim/relax_dependencies_lts_dlc2 branch November 4, 2025 11:02
deruyter92 added a commit to deruyter92/DeepLabCut that referenced this pull request Dec 5, 2025
Useful for investigating compatibility with different tensorflow versions (See PR DeepLabCut#3087 and DeepLabCut#3091)
deruyter92 added a commit to deruyter92/DeepLabCut that referenced this pull request Dec 5, 2025
Useful for investigating compatibility with different tensorflow versions (See PR DeepLabCut#3087 and DeepLabCut#3091)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

COLAB dependencies Pull requests that update a dependency file Installation Tensorflow 2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants