Skip to content

Fix _all_tables_exist#9075

Merged
harupy merged 1 commit intomlflow:masterfrom
harupy:fix_all_tables_exist
Jul 14, 2023
Merged

Fix _all_tables_exist#9075
harupy merged 1 commit intomlflow:masterfrom
harupy:fix_all_tables_exist

Conversation

@harupy
Copy link
Member

@harupy harupy commented Jul 14, 2023 •
edited
Loading

Related Issues/PRs

Resolve #9042

What changes are proposed in this pull request?

title

How is this patch tested?

  • Existing unit/integration tests
  • New unit/integration tests
  • Manual tests (describe details, including test results, below)
> mlflow server --backend-store-uri sqlite:///test.db
2023/07/14 19:40:54 INFO mlflow.store.db.utils: Creating initial MLflow database tables...
2023/07/14 19:40:54 INFO mlflow.store.db.utils: Updating database tables
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> 451aebb31d03, add metric step
INFO  [alembic.runtime.migration] Running upgrade 451aebb31d03 -> 90e64c465722, migrate user column to tags
INFO  [alembic.runtime.migration] Running upgrade 90e64c465722 -> 181f10493468, allow nulls for metric values
INFO  [alembic.runtime.migration] Running upgrade 181f10493468 -> df50e92ffc5e, Add Experiment Tags Table
INFO  [alembic.runtime.migration] Running upgrade df50e92ffc5e -> 7ac759974ad8, Update run tags with larger limit
INFO  [alembic.runtime.migration] Running upgrade 7ac759974ad8 -> 89d4b8295536, create latest metrics table
INFO  [89d4b8295536_create_latest_metrics_table_py] Migration complete!
INFO  [alembic.runtime.migration] Running upgrade 89d4b8295536 -> 2b4d017a5e9b, add model registry tables to db
INFO  [2b4d017a5e9b_add_model_registry_tables_to_db_py] Adding registered_models and model_versions tables to database.
INFO  [2b4d017a5e9b_add_model_registry_tables_to_db_py] Migration complete!
INFO  [alembic.runtime.migration] Running upgrade 2b4d017a5e9b -> cfd24bdc0731, Update run status constraint with killed
INFO  [alembic.runtime.migration] Running upgrade cfd24bdc0731 -> 0a8213491aaa, drop_duplicate_killed_constraint
INFO  [alembic.runtime.migration] Running upgrade 0a8213491aaa -> 728d730b5ebd, add registered model tags table
INFO  [alembic.runtime.migration] Running upgrade 728d730b5ebd -> 27a6a02d2cf1, add model version tags table
INFO  [alembic.runtime.migration] Running upgrade 27a6a02d2cf1 -> 84291f40a231, add run_link to model_version
INFO  [alembic.runtime.migration] Running upgrade 84291f40a231 -> a8c4a736bde6, allow nulls for run_id
INFO  [alembic.runtime.migration] Running upgrade a8c4a736bde6 -> 39d1c3be5f05, add_is_nan_constraint_for_metrics_tables_if_necessary
INFO  [alembic.runtime.migration] Running upgrade 39d1c3be5f05 -> c48cb773bb87, reset_default_value_for_is_nan_in_metrics_table_for_mysql
INFO  [alembic.runtime.migration] Running upgrade c48cb773bb87 -> bd07f7e963c5, create index on run_uuid
INFO  [alembic.runtime.migration] Running upgrade bd07f7e963c5 -> 0c779009ac13, add deleted_time field to runs table
INFO  [alembic.runtime.migration] Running upgrade 0c779009ac13 -> cc1f77228345, change param value length to 500
INFO  [alembic.runtime.migration] Running upgrade cc1f77228345 -> 97727af70f4d, Add creation_time and last_update_time to experiments table
INFO  [alembic.runtime.migration] Running upgrade 97727af70f4d -> 3500859a5d39, Add Model Aliases table
INFO  [alembic.runtime.migration] Running upgrade 3500859a5d39 -> 7f2a7d5fae7d, add datasets inputs input_tags tables
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
[2023-07-14 19:40:54 +0900] [175596] [INFO] Starting gunicorn 20.1.0
[2023-07-14 19:40:54 +0900] [175596] [INFO] Listening at: http://127.0.0.1:5000 (175596)
[2023-07-14 19:40:54 +0900] [175596] [INFO] Using worker: sync
[2023-07-14 19:40:54 +0900] [175601] [INFO] Booting worker with pid: 175601
[2023-07-14 19:40:54 +0900] [175603] [INFO] Booting worker with pid: 175603
[2023-07-14 19:40:54 +0900] [175606] [INFO] Booting worker with pid: 175606
[2023-07-14 19:40:54 +0900] [175607] [INFO] Booting worker with pid: 175607
[2023-07-14 19:41:07 +0900] [175596] [INFO] Handling signal: winch
^C[2023-07-14 19:41:10 +0900] [175596] [INFO] Handling signal: int
[2023-07-14 19:41:10 +0900] [175603] [INFO] Worker exiting (pid: 175603)
[2023-07-14 19:41:10 +0900] [175601] [INFO] Worker exiting (pid: 175601)
[2023-07-14 19:41:10 +0900] [175606] [INFO] Worker exiting (pid: 175606)
[2023-07-14 19:41:10 +0900] [175607] [INFO] Worker exiting (pid: 175607)
^C
Aborted!
[2023-07-14 19:41:10 +0900] [175596] [WARNING] Worker with pid 175601 was terminated due to signal 2
[2023-07-14 19:41:10 +0900] [175596] [WARNING] Worker with pid 175607 was terminated due to signal 2
[2023-07-14 19:41:10 +0900] [175596] [WARNING] Worker with pid 175606 was terminated due to signal 2
[2023-07-14 19:41:10 +0900] [175596] [WARNING] Worker with pid 175603 was terminated due to signal 2
[2023-07-14 19:41:10 +0900] [175596] [INFO] Shutting down: Master


> mlflow server --backend-store-uri sqlite:///test.db
[2023-07-14 19:41:12 +0900] [175979] [INFO] Starting gunicorn 20.1.0
[2023-07-14 19:41:12 +0900] [175979] [INFO] Listening at: http://127.0.0.1:5000 (175979)
[2023-07-14 19:41:12 +0900] [175979] [INFO] Using worker: sync
[2023-07-14 19:41:12 +0900] [175981] [INFO] Booting worker with pid: 175981
[2023-07-14 19:41:12 +0900] [175982] [INFO] Booting worker with pid: 175982
[2023-07-14 19:41:12 +0900] [175983] [INFO] Booting worker with pid: 175983
[2023-07-14 19:41:12 +0900] [175984] [INFO] Booting worker with pid: 175984

Does this PR change the documentation?

  • No. You can skip the rest of this section.
  • Yes. Make sure the changed pages / sections render correctly in the documentation preview.

Release Notes

Is this a user-facing change?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release notes for MLflow users.

(Details in 1-2 sentences. You can just refer to another PR with a description if this PR is part of a larger change.)

What component(s), interfaces, languages, and integrations does this PR affect?

Components

  • area/artifacts: Artifact stores and artifact logging
  • area/build: Build and test infrastructure for MLflow
  • area/docs: MLflow documentation pages
  • area/examples: Example code
  • area/gateway: AI Gateway service, Gateway client APIs, third-party Gateway integrations
  • area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
  • area/models: MLmodel format, model serialization/deserialization, flavors
  • area/recipes: Recipes, Recipe APIs, Recipe configs, Recipe Templates
  • area/projects: MLproject format, project running backends
  • area/scoring: MLflow Model server, model deployment tools, Spark UDFs
  • area/server-infra: MLflow Tracking server backend
  • area/tracking: Tracking Service, tracking client APIs, autologging

Interface

  • area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
  • area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
  • area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
  • area/windows: Windows support

Language

  • language/r: R APIs and clients
  • language/java: Java APIs and clients
  • language/new: Proposals for new client languages

Integrations

  • integrations/azure: Azure and Azure ML integrations
  • integrations/sagemaker: SageMaker integrations
  • integrations/databricks: Databricks integrations

How should the PR be classified in the release notes? Choose one:

  • rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
  • rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
  • rn/feature - A new user-facing feature worth mentioning in the release notes
  • rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
  • rn/documentation - A user-facing documentation change worth mentioning in the release notes

@mlflow-automation
Copy link
Contributor

mlflow-automation commented Jul 14, 2023

Documentation preview for a2e0087 will be available here when this CircleCI job completes successfully.

More info

@github-actions github-actions bot added the rn/none List under Small Changes in Changelogs. label Jul 14, 2023
BenWilson2
BenWilson2 approved these changes Jul 14, 2023
View reviewed changes
Copy link
Member

@BenWilson2 BenWilson2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@harupy harupy enabled auto-merge (squash) July 14, 2023 11:24
@BenWilson2
Copy link
Member

BenWilson2 commented Jul 14, 2023

Unrelated note (potentially for a good first issue) is that the Dataset and InputTag class constructor type signatures are wrong (not an issue as type checking is not performed, but those classes are not being passed str type in these definitions; they're being passed sqlalchemy.sql.schema.Column types)

@harupy harupy merged commit 47c58b5 into mlflow:master Jul 14, 2023
@harupy harupy deleted the fix_all_tables_exist branch July 14, 2023 12:17
BenWilson2 pushed a commit to BenWilson2/mlflow that referenced this pull request Jul 31, 2023
@harupy @BenWilson2
Fix _all_tables_exist (mlflow#9075)
5e58d5f 
Signed-off-by: harupy <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BenWilson2 BenWilson2 BenWilson2 approved these changes

Assignees

No one assigned

Labels

rn/none List under Small Changes in Changelogs.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[BUG] _all_tables_exist is out of date

3 participants

@harupy @mlflow-automation @BenWilson2