Skip to content

Conversation

@anna-charlotte
Copy link
Contributor

Goals:
allow nested attribute access in find(), which currently isn't supported yet. Nested access will be possible by '__'-separated access paths.

Example:

class InnerDoc(BaseDocument):
    title: str
    embedding: NdArray

class MyDoc(BaseDocument):
    inner: InnerDoc

query = MyDoc(inner=InnerDoc(title='query', embedding=np.random.randn((10))))
index = DocumentArray[MyDoc](
    [
        MyDoc(inner=InnerDoc(title=f'doc {i}', embedding=np.random.randn((10))))
        for i in range(10)
    ]
)

top_k, scores = find(
    index,
    query,
    embedding_field='inner__embedding',  # <----------- access path for nested access
    limit=7,
)
  • check and update documentation, if required. See guide

@anna-charlotte anna-charlotte mentioned this pull request Feb 24, 2023
47 tasks
@anna-charlotte anna-charlotte marked this pull request as ready for review February 27, 2023 08:40
emb = embedding_type._docarray_stack(emb_list)
elif isinstance(data, DocumentArrayStacked):
emb = getattr(data, embedding_field)
emb = next(AnyDocumentArray._traverse(data, embedding_field))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure to understand why it is next here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For DocumentArray stacked, the _traverse will yield one stacked tensor, by calling next() on it, I want to get that one tensor. I think this would be the same as calling list(AnyDocumentArray._traverse(data, embedding_field))[0].


def _get_field_type_by_access_path(
doc_type: Type['BaseDocument'], access_path: str
) -> Any:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-> Optional[Type]?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

applied!

anna-charlotte added 12 commits February 27, 2023 14:45
Signed-off-by: anna-charlotte <[email protected]>
Signed-off-by: anna-charlotte <[email protected]>
Signed-off-by: anna-charlotte <[email protected]>
Signed-off-by: anna-charlotte <[email protected]>
Signed-off-by: anna-charlotte <[email protected]>
@anna-charlotte anna-charlotte force-pushed the feat-find-nested-access branch from 68af091 to 8400ed4 Compare February 27, 2023 13:46
@github-actions
Copy link

📝 Docs are deployed on https://ft-feat-find-nested-access--jina-docs.netlify.app 🎉

@anna-charlotte anna-charlotte merged commit 28b96fe into feat-rewrite-v2 Feb 27, 2023
@anna-charlotte anna-charlotte deleted the feat-find-nested-access branch February 27, 2023 16:35
AsRaNi1 pushed a commit to AsRaNi1/docarray that referenced this pull request Feb 28, 2023
* test: add test for find on nested attribute

Signed-off-by: anna-charlotte <[email protected]>

* test: add test for nested attr in stacked da

Signed-off-by: anna-charlotte <[email protected]>

* fix: nested access

Signed-off-by: anna-charlotte <[email protected]>

* fix: mypy

Signed-off-by: anna-charlotte <[email protected]>

* fix: field type by access path for da

Signed-off-by: anna-charlotte <[email protected]>

* fix: clean up

Signed-off-by: anna-charlotte <[email protected]>

* fix: move get field type by access path

Signed-off-by: anna-charlotte <[email protected]>

* fix: imports

Signed-off-by: anna-charlotte <[email protected]>

* fix: apply suggestion and fix import

Signed-off-by: anna-charlotte <[email protected]>

* fix: apply suggestion from code review

Signed-off-by: anna-charlotte <[email protected]>

* fix: apply suggestion from code review

Signed-off-by: anna-charlotte <[email protected]>

* fix: apply suggestions

Signed-off-by: anna-charlotte <[email protected]>

---------

Signed-off-by: anna-charlotte <[email protected]>
Signed-off-by: Arnav Zutshi <[email protected]>
Jackmin801 pushed a commit that referenced this pull request Mar 2, 2023
* test: add test for find on nested attribute

Signed-off-by: anna-charlotte <[email protected]>

* test: add test for nested attr in stacked da

Signed-off-by: anna-charlotte <[email protected]>

* fix: nested access

Signed-off-by: anna-charlotte <[email protected]>

* fix: mypy

Signed-off-by: anna-charlotte <[email protected]>

* fix: field type by access path for da

Signed-off-by: anna-charlotte <[email protected]>

* fix: clean up

Signed-off-by: anna-charlotte <[email protected]>

* fix: move get field type by access path

Signed-off-by: anna-charlotte <[email protected]>

* fix: imports

Signed-off-by: anna-charlotte <[email protected]>

* fix: apply suggestion and fix import

Signed-off-by: anna-charlotte <[email protected]>

* fix: apply suggestion from code review

Signed-off-by: anna-charlotte <[email protected]>

* fix: apply suggestion from code review

Signed-off-by: anna-charlotte <[email protected]>

* fix: apply suggestions

Signed-off-by: anna-charlotte <[email protected]>

---------

Signed-off-by: anna-charlotte <[email protected]>
Jackmin801 pushed a commit that referenced this pull request Mar 2, 2023
* test: add test for find on nested attribute

Signed-off-by: anna-charlotte <[email protected]>

* test: add test for nested attr in stacked da

Signed-off-by: anna-charlotte <[email protected]>

* fix: nested access

Signed-off-by: anna-charlotte <[email protected]>

* fix: mypy

Signed-off-by: anna-charlotte <[email protected]>

* fix: field type by access path for da

Signed-off-by: anna-charlotte <[email protected]>

* fix: clean up

Signed-off-by: anna-charlotte <[email protected]>

* fix: move get field type by access path

Signed-off-by: anna-charlotte <[email protected]>

* fix: imports

Signed-off-by: anna-charlotte <[email protected]>

* fix: apply suggestion and fix import

Signed-off-by: anna-charlotte <[email protected]>

* fix: apply suggestion from code review

Signed-off-by: anna-charlotte <[email protected]>

* fix: apply suggestion from code review

Signed-off-by: anna-charlotte <[email protected]>

* fix: apply suggestions

Signed-off-by: anna-charlotte <[email protected]>

---------

Signed-off-by: anna-charlotte <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants