[DRAFT] feat: added client side metric instrumentation to read_rows and mutate_rows#16758
Conversation
There was a problem hiding this comment.
Code Review
This pull request integrates client-side metrics tracking into Bigtable data operations, including ReadRows and MutateRows, for both asynchronous and synchronous implementations. It introduces tracked_retry to capture operation metrics and refactors read_row to utilize operation classes directly, supported by extensive new system and unit tests. Feedback identifies opportunities to improve the efficiency of read_row by using next() or __anext__ on generators instead of converting them to lists, and suggests adding a type ignore comment in the synchronous code for consistency.
| try: | ||
| results = [a async for a in results_generator] | ||
| return results[0] | ||
| except IndexError: | ||
| return None |
There was a problem hiding this comment.
This implementation consumes the entire generator into a list just to get the first element. A more efficient and idiomatic way to get the first item from an async generator is to use anext.
| try: | |
| results = [a async for a in results_generator] | |
| return results[0] | |
| except IndexError: | |
| return None | |
| try: | |
| return await results_generator.__anext__() | |
| except StopAsyncIteration: | |
| return None |
| self._operation_metric.active_attempt.application_blocking_time_ns += ( | ||
| time.monotonic_ns() - block_time | ||
| ) |
There was a problem hiding this comment.
The corresponding async version of this code includes a # type: ignore comment here. To maintain consistency and prevent potential mypy errors, it would be good to add it here as well.
| self._operation_metric.active_attempt.application_blocking_time_ns += ( | |
| time.monotonic_ns() - block_time | |
| ) | |
| self._operation_metric.active_attempt.application_blocking_time_ns += ( | |
| time.monotonic_ns() - block_time | |
| ) # type: ignore |
References
- Using type: ignore is an acceptable trade-off to satisfy mypy when precise type hints are not cost-effective or to maintain consistency with other parts of the codebase.
| try: | ||
| results = [a for a in results_generator] | ||
| return results[0] | ||
| except IndexError: | ||
| return None |
There was a problem hiding this comment.
This implementation consumes the entire generator into a list just to get the first element. A more efficient and idiomatic way to get the first item from a generator is to use next().
| try: | |
| results = [a for a in results_generator] | |
| return results[0] | |
| except IndexError: | |
| return None | |
| try: | |
| return next(results_generator) | |
| except StopIteration: | |
| return None |
Migration of googleapis/python-bigtable#1256 to the monorepo
Builds off of #16712 to add instrumentation to read_rows and mutate_rows, along with the mutation batcher