feat(v2): add tensorflow support #1064

anna-charlotte · 2023-01-30T10:59:12Z

Signed-off-by: anna-charlotte [email protected]

Goals:

Add Tensorflow support to v2:

Add TF tensor type
- As of right now, I haven't found a solution on how to subclass a tf.Tensor (see here), so for now we weill store the tf.Tensor instance as an attribute of a TensorFlowTensor instance.
Add TF computational backend
check and update documentation, if required. See guide

docarray/typing/tensor/tensorflow_tensor.py

github-actions · 2023-01-30T16:33:34Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-01-31T08:15:44Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-01-31T12:20:11Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

docarray/computation/abstract_numpy_based_backend.py

github-actions · 2023-02-01T14:09:19Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-02-01T18:34:41Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-02-02T09:24:26Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-02-02T12:37:22Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-02-02T13:15:49Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

samsja · 2023-02-02T14:08:17Z

docarray/computation/tensorflow_backend.py

+    return t.tensor
+
+
+class TensorFlowCompBackend(AbstractNumpyBasedBackend[TensorFlowTensor]):


general comment about the class, to me it looks good I like that we rely on tnp as much as we can.

Maybe norm_left, norm_right should be named differenlty

norm_left could be : cast_output
norm_right could be: get_tensor

this will make the code clearer

maybe @JohannesMessner as a better idea for the name.

docarray/typing/tensor/tensorflow_tensor.py

samsja

I added few comments

github-actions · 2023-02-03T16:05:50Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

…rflow-support

github-actions · 2023-02-08T07:42:55Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

Signed-off-by: anna-charlotte <[email protected]>

github-actions · 2023-02-08T08:40:34Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

Signed-off-by: anna-charlotte <[email protected]>

github-actions · 2023-02-08T08:47:09Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

Signed-off-by: anna-charlotte <[email protected]>

github-actions · 2023-02-08T08:48:39Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

Signed-off-by: anna-charlotte <[email protected]>

github-actions · 2023-02-08T09:14:24Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

JohannesMessner · 2023-02-08T10:18:53Z

docarray/array/array_stacked.py

+try:
+    import tensorflow as tf  # type: ignore
+
+    from docarray.typing import TensorFlowTensor


for torch i moved this thing to a helper in utils, so this check only has to be done once globally. Can we do the same for tf?

yes, saw that and started doing this in the TF embedding/video/audio PR, so i'll do this refactor there if that's fine with you @JohannesMessner

github-actions · 2023-02-08T10:55:48Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

JoanFM · 2023-02-08T11:37:28Z

README.md

+        return inputs
+```
+
+Much nicer, don't you think? One main difference to using DocArray with PyTorch, is that when using TensorFlowTensor's, you have to access it's `.tensor` attribute directly, as it can be seen in `.forward_podcast()` above. This is due to the fact that while `TorchTensor` is a subclass of `torch.Tensor`, `TensorFlowTensor` is not a subclass of `tf.Tensor` but instead stores a `tf.Tensor` in its `.tensor` attribute.


It may be good to explain why this is the case?

Why is this different?

Why is this different?
@anna-charlotte you can probably add due to a technical limitation on tf.Tensor

github-actions · 2023-02-08T12:02:13Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

samsja · 2023-02-08T10:56:37Z

README.md

 schema definition (see below). Everything handles in a pythonic manner by relying on type hints.


+## Coming from TensorFlow


this part is too big IMO. We just need to show that there is a (tiny ?) difference and that you need to access tensor.tensor. No need to show the full example

samsja · 2023-02-08T13:17:13Z

README.md

+        return inputs
+```
+
+Much nicer, don't you think? One main difference to using DocArray with PyTorch, is that when using TensorFlowTensor's, you have to access it's `.tensor` attribute directly, as it can be seen in `.forward_podcast()` above. This is due to the fact that while `TorchTensor` is a subclass of `torch.Tensor`, `TensorFlowTensor` is not a subclass of `tf.Tensor` but instead stores a `tf.Tensor` in its `.tensor` attribute.


Why is this different?
@anna-charlotte you can probably add due to a technical limitation on tf.Tensor

Signed-off-by: anna-charlotte <[email protected]>

github-actions · 2023-02-08T13:58:16Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-02-08T14:00:00Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions · 2023-02-08T14:03:45Z

📝 Docs are deployed on https://ft-feat-tensorflow-support--jina-docs.netlify.app 🎉

github-actions · 2023-02-08T14:08:53Z

📝 Docs are deployed on https://ft-feat-tensorflow-support--jina-docs.netlify.app 🎉

JohannesMessner

LGTM

anna-charlotte linked an issue Jan 30, 2023 that may be closed by this pull request

v2: Add Tensforflow support #1057

Closed

3 tasks

github-actions bot added size/m area/core area/typing labels Jan 30, 2023

JohannesMessner reviewed Jan 30, 2023

View reviewed changes

docarray/typing/tensor/tensorflow_tensor.py Outdated Show resolved Hide resolved

docarray/typing/tensor/tensorflow_tensor.py Outdated Show resolved Hide resolved

github-actions bot added area/testing size/xl and removed size/m labels Jan 30, 2023

anna-charlotte commented Jan 31, 2023

View reviewed changes

docarray/computation/abstract_numpy_based_backend.py Outdated Show resolved Hide resolved

anna-charlotte commented Jan 31, 2023

View reviewed changes

docarray/computation/abstract_numpy_based_backend.py Outdated Show resolved Hide resolved

github-actions bot added size/l and removed size/xl labels Jan 31, 2023

anna-charlotte mentioned this pull request Feb 1, 2023

Meta: DocArray v2 Roadmap #780

Closed

47 tasks

github-actions bot added size/xl and removed size/l labels Feb 1, 2023

anna-charlotte force-pushed the feat-tensorflow-support branch from e410061 to efac716 Compare February 1, 2023 18:34

samsja reviewed Feb 2, 2023

View reviewed changes

docarray/typing/tensor/tensorflow_tensor.py Show resolved Hide resolved

samsja reviewed Feb 2, 2023

View reviewed changes

anna-charlotte force-pushed the feat-tensorflow-support branch from 5352804 to 494a9eb Compare February 3, 2023 16:05

github-actions bot added the component/array label Feb 3, 2023

Merge remote-tracking branch 'origin/feat-rewrite-v2' into feat-tenso…

4905526

…rflow-support

docs: update readme.md

c2aa0b1

Signed-off-by: anna-charlotte <[email protected]>

fix: remove n dim from abstract method instead use comp be

81a2540

Signed-off-by: anna-charlotte <[email protected]>

fix: remove proto mark, because only test for proto 3 here

838955a

Signed-off-by: anna-charlotte <[email protected]>

fix: tf set item and add tests

09dd6a9

Signed-off-by: anna-charlotte <[email protected]>

JohannesMessner requested changes Feb 8, 2023

View reviewed changes

Merge branch 'feat-rewrite-v2' into feat-tensorflow-support

5daa511

anna-charlotte marked this pull request as ready for review February 8, 2023 11:19

JoanFM reviewed Feb 8, 2023

View reviewed changes

Merge branch 'feat-rewrite-v2' into feat-tensorflow-support

6c36d25

samsja reviewed Feb 8, 2023

View reviewed changes

docs: update tf section in readme.md

85183ec

Signed-off-by: anna-charlotte <[email protected]>

anna-charlotte closed this Feb 8, 2023

anna-charlotte reopened this Feb 8, 2023

samsja approved these changes Feb 8, 2023

View reviewed changes

JohannesMessner approved these changes Feb 8, 2023

View reviewed changes

samsja merged commit 6b0d13c into feat-rewrite-v2 Feb 8, 2023

samsja deleted the feat-tensorflow-support branch February 8, 2023 14:27

anna-charlotte mentioned this pull request Feb 20, 2023

DocArray v2 third alpha release note #1153

Closed

		return t.tensor


		class TensorFlowCompBackend(AbstractNumpyBasedBackend[TensorFlowTensor]):

		schema definition (see below). Everything handles in a pythonic manner by relying on type hints.


		## Coming from TensorFlow

feat(v2): add tensorflow support #1064

feat(v2): add tensorflow support #1064

Uh oh!

Conversation

anna-charlotte commented Jan 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jan 30, 2023

Uh oh!

github-actions bot commented Jan 31, 2023

Uh oh!

github-actions bot commented Jan 31, 2023

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Feb 1, 2023

Uh oh!

github-actions bot commented Feb 1, 2023

Uh oh!

github-actions bot commented Feb 2, 2023

Uh oh!

github-actions bot commented Feb 2, 2023

Uh oh!

github-actions bot commented Feb 2, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

samsja left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 3, 2023

Uh oh!

github-actions bot commented Feb 8, 2023

Uh oh!

github-actions bot commented Feb 8, 2023

Uh oh!

github-actions bot commented Feb 8, 2023

Uh oh!

github-actions bot commented Feb 8, 2023

Uh oh!

github-actions bot commented Feb 8, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 8, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 8, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 8, 2023

Uh oh!

github-actions bot commented Feb 8, 2023

Uh oh!

github-actions bot commented Feb 8, 2023

Uh oh!

github-actions bot commented Feb 8, 2023

Uh oh!

JohannesMessner left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

anna-charlotte commented Jan 30, 2023 •

edited

Loading