Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: (observability) add spans for BatchTransaction and Table #2115

Conversation

odeke-em
Copy link
Contributor

This change is part of a series of changes to add
OpenTelemetry traces, focused on BatchTransaction and Table.

While here, made the tests for sessionPool spans much more precise to avoid flakes.

Updates #2079
Built from PR #2087
Updates #2114

@odeke-em odeke-em requested review from a team as code owners September 17, 2024 13:09
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: spanner Issues related to the googleapis/nodejs-spanner API. labels Sep 17, 2024
src/table.ts Outdated Show resolved Hide resolved
@surbhigarg92
Copy link
Contributor

@odeke-em I don't see any tests for these spans.

@surbhigarg92 surbhigarg92 changed the title observability: trace BatchTransaction and Table feat: (observability) add spans for BatchTransaction and Table Sep 19, 2024
@odeke-em odeke-em force-pushed the trace-instrument-BatchTransaction+Table branch from e74c73e to ea6d7f2 Compare September 19, 2024 13:11
@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Sep 19, 2024
@odeke-em
Copy link
Contributor Author

Hey @surbhigarg92 I hadn't uploaded some of them as I created this PR early morning at around 3AM before bed and in interludes. I've added comprehensive tests, please take a look.

@odeke-em odeke-em force-pushed the trace-instrument-BatchTransaction+Table branch from ea6d7f2 to 38708a8 Compare September 19, 2024 13:15
test/batch-transaction.ts Outdated Show resolved Hide resolved
@@ -285,6 +286,7 @@ export class Snapshot extends EventEmitter {
queryOptions?: IQueryOptions;
resourceHeader_: {[k: string]: string};
requestOptions?: Pick<IRequestOptions, 'transactionTag'>;
observabilityOptions?: ObservabilityOptions;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

observabilityOptions should be injected by customers from SpannerOptions which should set this property. Do you plan to do that in separate PR ?
I would recommend to add that in the same PR, so that we can test batch-transaction and mutation flow end to end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I shall have it plumbed in progressively as we get there. The reason being that this PR is bound to balloon if I add it straight from the top inside new Spanner({observabilityOptions: opts}). Just the way for each part I have been adding it in and testing.

test/batch-transaction.ts Outdated Show resolved Hide resolved
test/batch-transaction.ts Outdated Show resolved Hide resolved
test/batch-transaction.ts Outdated Show resolved Hide resolved
test/batch-transaction.ts Outdated Show resolved Hide resolved
test/batch-transaction.ts Outdated Show resolved Hide resolved
test/batch-transaction.ts Outdated Show resolved Hide resolved
test/batch-transaction.ts Outdated Show resolved Hide resolved
1,
'Exactly 1 span should have been exported'
);
assert.strictEqual(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did we remove this check from here ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the testing environment, when ran isolated, the checks pass but when ran as a whole this test flakes here and there, a potential problem with OpenTelemetry. Besides, the point of the code was to ensure that the parent span actually has the required annotations which we do have and now it is very well cut out just for that.

@odeke-em odeke-em force-pushed the trace-instrument-BatchTransaction+Table branch 3 times, most recently from db71593 to 7357d35 Compare September 20, 2024 04:09
@odeke-em odeke-em force-pushed the trace-instrument-BatchTransaction+Table branch from 7357d35 to 23032f9 Compare September 20, 2024 04:16
observability-test/batch-transaction.ts Outdated Show resolved Hide resolved
@@ -1367,34 +1352,12 @@ describe('SessionPool', () => {
});

describe('trace annotations on active span', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe for separate PR. But It will be good to extract these tests in observability-test folder

This change is part of a series of changes to add
OpenTelemetry traces, focused on BatchTransaction and Table.

While here, made the tests for sessionPool spans much more
precise to avoid flakes.

Updates googleapis#2079
Built from PR googleapis#2087
Updates googleapis#2114
@odeke-em odeke-em force-pushed the trace-instrument-BatchTransaction+Table branch from 23032f9 to 6adf870 Compare September 20, 2024 05:35
@surbhigarg92 surbhigarg92 added kokoro:run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels Sep 20, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Sep 20, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Sep 20, 2024
@surbhigarg92 surbhigarg92 added the automerge Merge the pull request once unit tests and other checks pass. label Sep 20, 2024
@gcf-merge-on-green gcf-merge-on-green bot merged commit d51aae9 into googleapis:main Sep 20, 2024
16 checks passed
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Sep 20, 2024
gcf-merge-on-green bot pushed a commit that referenced this pull request Oct 30, 2024
🤖 I have created a release *beep* *boop*
---


## [7.15.0](https://togithub.com/googleapis/nodejs-spanner/compare/v7.14.0...v7.15.0) (2024-10-30)


### Features

* (observability, samples): add tracing end-to-end sample ([#2130](https://togithub.com/googleapis/nodejs-spanner/issues/2130)) ([66d99e8](https://togithub.com/googleapis/nodejs-spanner/commit/66d99e836cd2bfbb3b0f78980ec2b499f9e5e563))
* (observability) add spans for BatchTransaction and Table ([#2115](https://togithub.com/googleapis/nodejs-spanner/issues/2115)) ([d51aae9](https://togithub.com/googleapis/nodejs-spanner/commit/d51aae9c9c3c0e6319d81c2809573ae54675acf3)), closes [#2114](https://togithub.com/googleapis/nodejs-spanner/issues/2114)
* (observability) Add support for OpenTelemetry traces and allow observability options to be passed.  ([#2131](https://togithub.com/googleapis/nodejs-spanner/issues/2131)) ([5237e11](https://togithub.com/googleapis/nodejs-spanner/commit/5237e118befb4b7fe4aea76a80a91e822d7a22e4)), closes [#2079](https://togithub.com/googleapis/nodejs-spanner/issues/2079)
* (observability) propagate database name for every span generated to aid in quick debugging ([#2155](https://togithub.com/googleapis/nodejs-spanner/issues/2155)) ([0342e74](https://togithub.com/googleapis/nodejs-spanner/commit/0342e74721a0684d8195a6299c3a634eefc2b522))
* (observability) trace Database.batchCreateSessions + SessionPool.createSessions ([#2145](https://togithub.com/googleapis/nodejs-spanner/issues/2145)) ([f489c94](https://togithub.com/googleapis/nodejs-spanner/commit/f489c9479fa5402f0c960cf896fd3be0e946f182))
* (observability): trace Database.runPartitionedUpdate ([#2176](https://togithub.com/googleapis/nodejs-spanner/issues/2176)) ([701e226](https://togithub.com/googleapis/nodejs-spanner/commit/701e22660d5ac9f0b3e940ad656b9ca6c479251d)), closes [#2079](https://togithub.com/googleapis/nodejs-spanner/issues/2079)
* (observability): trace Database.runTransactionAsync ([#2167](https://togithub.com/googleapis/nodejs-spanner/issues/2167)) ([d0fe178](https://togithub.com/googleapis/nodejs-spanner/commit/d0fe178623c1c48245d11bcea97fcd340b6615af)), closes [#207](https://togithub.com/googleapis/nodejs-spanner/issues/207)
* Allow multiple KMS keys to create CMEK database/backup ([#2099](https://togithub.com/googleapis/nodejs-spanner/issues/2099)) ([51bc8a7](https://togithub.com/googleapis/nodejs-spanner/commit/51bc8a7445ab8b3d2239493b69d9c271c1086dde))
* **observability:** Fix bugs found from product review + negative cases ([#2158](https://togithub.com/googleapis/nodejs-spanner/issues/2158)) ([cbc86fa](https://togithub.com/googleapis/nodejs-spanner/commit/cbc86fa80498af6bd745eebb9443612936e26d4e))
* **observability:** Trace Database methods ([#2119](https://togithub.com/googleapis/nodejs-spanner/issues/2119)) ([1f06871](https://togithub.com/googleapis/nodejs-spanner/commit/1f06871f7aca386756e8691013602b069697bb87)), closes [#2114](https://togithub.com/googleapis/nodejs-spanner/issues/2114)
* **observability:** Trace Database.batchWriteAtLeastOnce ([#2157](https://togithub.com/googleapis/nodejs-spanner/issues/2157)) ([2a19ef1](https://togithub.com/googleapis/nodejs-spanner/commit/2a19ef1af4f6fd1b81d08afc15db76007859a0b9)), closes [#2079](https://togithub.com/googleapis/nodejs-spanner/issues/2079)
* **observability:** Trace Transaction ([#2122](https://togithub.com/googleapis/nodejs-spanner/issues/2122)) ([a464bdb](https://togithub.com/googleapis/nodejs-spanner/commit/a464bdb5cbb7856b7a08dac3ff48132948b65792)), closes [#2114](https://togithub.com/googleapis/nodejs-spanner/issues/2114)


### Bug Fixes

* Exact staleness timebound ([#2143](https://togithub.com/googleapis/nodejs-spanner/issues/2143)) ([f01516e](https://togithub.com/googleapis/nodejs-spanner/commit/f01516ec6ba44730622cfb050c52cd93f30bba7a)), closes [#2129](https://togithub.com/googleapis/nodejs-spanner/issues/2129)
* GetMetadata for Session ([#2124](https://togithub.com/googleapis/nodejs-spanner/issues/2124)) ([2fd63ac](https://togithub.com/googleapis/nodejs-spanner/commit/2fd63acb87ce06a02d7fdfa78d836dbd7ad59a26)), closes [#2123](https://togithub.com/googleapis/nodejs-spanner/issues/2123)

---
This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the googleapis/nodejs-spanner API. size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants