Skip to content

Improve parser#209

Merged
sunpe merged 2 commits intotaosdata:mainfrom
hadrianl:improve_parser
Aug 9, 2023
Merged

Improve parser#209
sunpe merged 2 commits intotaosdata:mainfrom
hadrianl:improve_parser

Conversation

@hadrianl
Copy link
Copy Markdown
Contributor

@hadrianl hadrianl commented Aug 7, 2023

resolved #204

This PR improve the performance of taos query[c -> python parser] :

  • reduce cast, memory copy and object creation
  • convert timezone once when set_tz, not everytime _convert_XXX_to_datetime
  • totally, it take only 10%-20% cpu time when dealing with binary, nchar and timestamp

further more

  • taos_is_null had been called so many time per query
  • a vector version of taos_is_null from libtaos is needed to improve this, just like taos_get_column_data_offset
  • or maybe, introducing the third party libraries such as Cython to improve this

@sangshuduo sangshuduo requested a review from sunpe August 7, 2023 02:43
sangshuduo
sangshuduo approved these changes Aug 7, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@sangshuduo sangshuduo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Aug 7, 2023

Codecov Report

Merging #209 (56b8ca1) into main (9dad4b5) will decrease coverage by 0.04%.
The diff coverage is 75.00%.

@@            Coverage Diff             @@
##             main     #209      +/-   ##
==========================================
- Coverage   81.37%   81.33%   -0.04%     
==========================================
  Files          20       20              
  Lines        2652     2652              
==========================================
- Hits         2158     2157       -1     
- Misses        494      495       +1
Files Changed Coverage Δ
taos/field.py 79.75% <25.00%> (-0.37%) ⬇️
taos/field_v3.py 93.75% <100.00%> (-0.16%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

sunpe
sunpe approved these changes Aug 9, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@sunpe sunpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sunpe sunpe merged commit 28b4505 into taosdata:main Aug 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@sangshuduo sangshuduo sangshuduo approved these changes

@sunpe sunpe sunpe approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

taos_fetch_block _v3 is slow

4 participants

@hadrianl @codecov-commenter @sangshuduo @sunpe