feat: make compatible based type with fastapi #809

samsja · 2022-11-17T16:10:49Z

Context

The goal of this PR is make our Document model compatible with FastAPI.

What this PR do:

Mainly add json dump to types

allow to do

from fastapi import FastAPI

from docarray import Document, Image, Text
from docarray.typing import Tensor


class InputDoc(Document):
    text: str

class OutputDoc(Document):
    embedding_clip: Tensor
    embedding_bert: Tensor

input_doc = InputDoc(text='hello')

app = FastAPI()

@app.post("/doc/", response_model=OutputDoc)
async def create_item(doc: InputDoc) -> OutputDoc:
    ## call my fancy model to generate the embeddings
    return OutputDoc(
        embedding_clip=np.zeros((100, 1)), embedding_bert=np.zeros((100, 1))
    )

and it is working as well with our custom types

Signed-off-by: Sami Jaghouar <[email protected]>

* feat: add tensor type for ndarray * fix: fix mypy typing * feat: torch tensor type Signed-off-by: Johannes Messner <[email protected]> * fix: protobuf for pytorch type Signed-off-by: Johannes Messner <[email protected]> * ci: install all extras in the ci Signed-off-by: Johannes Messner <[email protected]> * refactor: make nice looking * docs: update docarray/typing/tensor/torch_tensor.py Co-authored-by: samsja <[email protected]> Signed-off-by: Johannes Messner <[email protected]> * refactor: code style Signed-off-by: Johannes Messner <[email protected]> * fix: black and mypy Signed-off-by: Johannes Messner <[email protected]> * fix: suppress mypy import error * ci: fix ci install Signed-off-by: Johannes Messner <[email protected]> Signed-off-by: Johannes Messner <[email protected]> Signed-off-by: Johannes Messner <[email protected]> Co-authored-by: samsja <[email protected]>

Signed-off-by: Sami Jaghouar <[email protected]>

docarray/document/io/json.py

docarray/typing/tensor/tensor.py

docarray/typing/tensor/torch_tensor.py

Signed-off-by: Sami Jaghouar <[email protected]>

docarray/typing/tensor/torch_tensor.py

Co-authored-by: Johannes Messner <[email protected]> Signed-off-by: samsja <[email protected]>

Signed-off-by: Sami Jaghouar <[email protected]>

* feat: allow da bulk access to return da for document Signed-off-by: Sami Jaghouar <[email protected]> * fix: fix mypy type pb Signed-off-by: Sami Jaghouar <[email protected]> * fix: add link to the mypy issue Signed-off-by: Sami Jaghouar <[email protected]> * fix: remove useless list type hint Signed-off-by: Sami Jaghouar <[email protected]> * feat: torch tensor type (#800) * feat: add tensor type for ndarray * fix: fix mypy typing * feat: torch tensor type Signed-off-by: Johannes Messner <[email protected]> * fix: protobuf for pytorch type Signed-off-by: Johannes Messner <[email protected]> * ci: install all extras in the ci Signed-off-by: Johannes Messner <[email protected]> * refactor: make nice looking * docs: update docarray/typing/tensor/torch_tensor.py Co-authored-by: samsja <[email protected]> Signed-off-by: Johannes Messner <[email protected]> * refactor: code style Signed-off-by: Johannes Messner <[email protected]> * fix: black and mypy Signed-off-by: Johannes Messner <[email protected]> * fix: suppress mypy import error * ci: fix ci install Signed-off-by: Johannes Messner <[email protected]> Signed-off-by: Johannes Messner <[email protected]> Signed-off-by: Johannes Messner <[email protected]> Co-authored-by: samsja <[email protected]> * feat: add fastapi to dependency Signed-off-by: Sami Jaghouar <[email protected]> * feat(wip): add fake method to dump tensor to json Signed-off-by: Sami Jaghouar <[email protected]> * feat(wip): add fastapi test Signed-off-by: Sami Jaghouar <[email protected]> * feat: add json dump for type and document Signed-off-by: Sami Jaghouar <[email protected]> * feat: add json compatible with orjson Signed-off-by: Sami Jaghouar <[email protected]> * refactor: clean tests Signed-off-by: Sami Jaghouar <[email protected]> * fix: remove duplicate Signed-off-by: Sami Jaghouar <[email protected]> * fix: better json schema for tensor Signed-off-by: Sami Jaghouar <[email protected]> * fix: fix fast api test Signed-off-by: Sami Jaghouar <[email protected]> * refactor: move to json test to integration Signed-off-by: Sami Jaghouar <[email protected]> * fix: json laod from tensor type now working Signed-off-by: Sami Jaghouar <[email protected]> * fix: add tensor to fastapi test Signed-off-by: Sami Jaghouar <[email protected]> * fix: add new fastapi test Signed-off-by: Sami Jaghouar <[email protected]> * fix: fix mypy Signed-off-by: Sami Jaghouar <[email protected]> * feat: add more testing for text uri about json Signed-off-by: Sami Jaghouar <[email protected]> * fix: fix default orson not returning Signed-off-by: Sami Jaghouar <[email protected]> * fix: apply johannes suggestion on docstring Signed-off-by: Sami Jaghouar <[email protected]> * fix: does not perform copy anymore on torch tensor unwrap Signed-off-by: Sami Jaghouar <[email protected]> * fix: add johannes suggestion Co-authored-by: Johannes Messner <[email protected]> Signed-off-by: samsja <[email protected]> * fix: fix ruff line lenght Signed-off-by: Sami Jaghouar <[email protected]> * fix: fix mypy pb Signed-off-by: Sami Jaghouar <[email protected]> Signed-off-by: Sami Jaghouar <[email protected]> Signed-off-by: Johannes Messner <[email protected]> Signed-off-by: Johannes Messner <[email protected]> Signed-off-by: samsja <[email protected]> Co-authored-by: Johannes Messner <[email protected]>

samsja marked this pull request as draft November 17, 2022 16:10

samsja changed the base branch from main to feat-rewrite-v2 November 17, 2022 16:11

samsja and others added 8 commits November 17, 2022 18:04

feat: allow da bulk access to return da for document

77a2055

Signed-off-by: Sami Jaghouar <[email protected]>

fix: fix mypy type pb

bd6d128

Signed-off-by: Sami Jaghouar <[email protected]>

fix: add link to the mypy issue

d778cec

Signed-off-by: Sami Jaghouar <[email protected]>

fix: remove useless list type hint

717e7ee

Signed-off-by: Sami Jaghouar <[email protected]>

feat: add fastapi to dependency

e5d678c

Signed-off-by: Sami Jaghouar <[email protected]>

feat(wip): add fake method to dump tensor to json

414a04c

Signed-off-by: Sami Jaghouar <[email protected]>

feat(wip): add fastapi test

42c6fe4

Signed-off-by: Sami Jaghouar <[email protected]>

samsja force-pushed the feat-v2-fastapi-test branch from a652aaa to 42c6fe4 Compare November 17, 2022 17:04

feat: add json dump for type and document

56497b9

Signed-off-by: Sami Jaghouar <[email protected]>

samsja force-pushed the feat-v2-fastapi-test branch from 3a894fa to 56497b9 Compare November 18, 2022 14:23

samsja added 8 commits November 21, 2022 20:09

feat: add json compatible with orjson

945a72a

Signed-off-by: Sami Jaghouar <[email protected]>

refactor: clean tests

83236cd

Signed-off-by: Sami Jaghouar <[email protected]>

fix: remove duplicate

f1a4d7b

Signed-off-by: Sami Jaghouar <[email protected]>

fix: better json schema for tensor

c983246

Signed-off-by: Sami Jaghouar <[email protected]>

fix: fix fast api test

3873a9a

Signed-off-by: Sami Jaghouar <[email protected]>

refactor: move to json test to integration

51a402d

Signed-off-by: Sami Jaghouar <[email protected]>

fix: json laod from tensor type now working

2e92866

Signed-off-by: Sami Jaghouar <[email protected]>

fix: add tensor to fastapi test

606134b

Signed-off-by: Sami Jaghouar <[email protected]>

samsja force-pushed the feat-v2-fastapi-test branch from 100916c to 87bab05 Compare November 22, 2022 13:43

fix: add new fastapi test

731feff

Signed-off-by: Sami Jaghouar <[email protected]>

samsja force-pushed the feat-v2-fastapi-test branch from 87bab05 to 731feff Compare November 22, 2022 13:50

samsja added 3 commits November 22, 2022 15:28

Merge branch 'feat-rewrite-v2' into feat-v2-fastapi-test

e114b91

fix: fix mypy

061b05e

Signed-off-by: Sami Jaghouar <[email protected]>

feat: add more testing for text uri about json

94c0069

Signed-off-by: Sami Jaghouar <[email protected]>

samsja marked this pull request as ready for review November 22, 2022 14:47

dongxiang123 approved these changes Nov 22, 2022

View reviewed changes

JohannesMessner requested changes Nov 22, 2022

View reviewed changes

samsja added 3 commits November 22, 2022 16:56

fix: fix default orson not returning

e04285a

Signed-off-by: Sami Jaghouar <[email protected]>

fix: apply johannes suggestion on docstring

b8de8bc

Signed-off-by: Sami Jaghouar <[email protected]>

fix: does not perform copy anymore on torch tensor unwrap

19ffb0b

Signed-off-by: Sami Jaghouar <[email protected]>

JohannesMessner reviewed Nov 22, 2022

View reviewed changes

docarray/typing/tensor/torch_tensor.py Outdated Show resolved Hide resolved

samsja and others added 3 commits November 22, 2022 17:52

fix: add johannes suggestion

0b7dac7

Co-authored-by: Johannes Messner <[email protected]> Signed-off-by: samsja <[email protected]>

fix: fix ruff line lenght

a334c65

Signed-off-by: Sami Jaghouar <[email protected]>

fix: fix mypy pb

909ee74

Signed-off-by: Sami Jaghouar <[email protected]>

JohannesMessner approved these changes Nov 23, 2022

View reviewed changes

JohannesMessner merged commit 42c7cdf into feat-rewrite-v2 Nov 23, 2022

JohannesMessner deleted the feat-v2-fastapi-test branch November 23, 2022 08:07

samsja mentioned this pull request Nov 23, 2022

Make compatible with fastapi #838

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: make compatible based type with fastapi #809

feat: make compatible based type with fastapi #809

Uh oh!

samsja commented Nov 17, 2022 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

feat: make compatible based type with fastapi #809

feat: make compatible based type with fastapi #809

Uh oh!

Conversation

samsja commented Nov 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

What this PR do:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

samsja commented Nov 17, 2022 •

edited

Loading