Add systests for read/query w/ concurrent updates. by tseaver · Pull Request #3632 · googleapis/google-cloud-python

tseaver · 2017-07-19T21:12:38Z

No description provided.

dhermes

LGTM pending green CI (should you push a branch to the GCP remote so we can see the system tests?)

Sign in to view

        self._check_row_data(rows)

+    def _transaction_concurrency_helper(self, unit_of_work, pkey):
+        import threading


Sign in to view

+                [[pkey, INITIAL_VALUE]])
+
+        # We don't want to run the threads' transactions in the current
+        # session, which would fail.


Sign in to view

+        self.assertEqual(len(rows), 1)
+        pkey, value = rows[0]
+        transaction.update(
+                self.COUNTERS_TABLE,


tseaver · 2017-07-19T21:34:26Z

Test failures look unrelated to our code :<

theacodes · 2017-07-19T21:35:53Z

@tseaver I'm working on it, it's a problem in our base image.

Sign in to view

+        retry = RetryInstanceState(_has_all_ddl)
+        retry(self._db.reload)()
+
+        class TestException(Exception):


Sign in to view

+
+        with session.batch() as batch:
+            batch.delete(self.TABLE, self.ALL)
+


theacodes · 2017-07-19T21:57:10Z

@tseaver I'm working on it, it's a problem in our base image.

Base image fixed, any test failures are no longer my fault. :)

Sign in to view

+        rows = list(transaction.read(self.TABLE, self.COLUMNS, self.ALL))
+        self.assertEqual(len(rows), 0)
+        transaction.insert(self.TABLE, self.COLUMNS, self.ROW_DATA)
+        raise TestException()


dhermes · 2017-07-19T22:08:45Z

@tseaver I presume https://circleci.com/gh/GoogleCloudPlatform/google-cloud-python/2453 is the build in the GCP remote?

dhermes · 2017-07-19T22:10:37Z

@tseaver Rename your custom exception? pytest is trying to collect it:

tests/system/test_system.py::TestException
  cannot collect test class 'TestException' because it has a __init__ constructor

-- Docs: http://doc.pytest.org/en/latest/warnings.html

tseaver · 2017-07-19T22:45:57Z

Ugh, failures are now due to BQ system test flakiness, Why are we running tests for the other APIs again?

dhermes · 2017-07-19T23:07:30Z

Why are we running tests for the other APIs again?

Because no on (you or I included) has written tooling to make the build aware of what it should be building. The relevant code is here https://github.com/GoogleCloudPlatform/google-cloud-python/blob/3d9461b91963fcc6e6a864f6f0eacad3d92bbf2d/test_utils/scripts/get_target_packages.py but IMO it's not worth it since we're breaking up the monorepo soon.

tseaver · 2017-07-19T23:22:28Z

Breaking up the monorepo doesn't seem to be high up on anybody's priority list ATM.

* Also add systest for user exception aborting transaction.

tseaver added api: spanner Issues related to the Spanner API. testing labels Jul 19, 2017

tseaver requested review from dhermes and lukesneeringer July 19, 2017 21:12

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jul 19, 2017

dhermes reviewed Jul 19, 2017

View reviewed changes

Comment thread spanner/tests/system/test_system.py Outdated

retry = RetryInstanceState(_has_all_ddl)

retry(self._db.reload)()

class TestException(Exception):

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

dhermes reviewed Jul 19, 2017

View reviewed changes

Comment thread spanner/tests/system/test_system.py

with session.batch() as batch:

batch.delete(self.TABLE, self.ALL)

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

dhermes reviewed Jul 19, 2017

View reviewed changes

Comment thread spanner/tests/system/test_system.py Outdated

rows = list(transaction.read(self.TABLE, self.COLUMNS, self.ALL))

self.assertEqual(len(rows), 0)

transaction.insert(self.TABLE, self.COLUMNS, self.ROW_DATA)

raise TestException()

This comment was marked as spam.

Sign in to view

tseaver added 5 commits July 20, 2017 18:02

Add systets for read/query w/ concurrent updates.

0a60bda

Add systest for user exception aborting transaction.

e2c2946

Address review comments.

db86cca

Address more review feedback.

561e8f5

Mollify py.test.

e50e31a

tseaver merged commit f8758df into googleapis:master Jul 20, 2017

tseaver deleted the spanner-systests-read-query-concurrent-update branch July 20, 2017 22:18

landrito pushed a commit to landrito/google-cloud-python that referenced this pull request Aug 21, 2017

Add systests for read/query w/ concurrent updates. (googleapis#3632)

16e6ef3

* Also add systest for user exception aborting transaction.

landrito pushed a commit to landrito/google-cloud-python that referenced this pull request Aug 22, 2017

Add systests for read/query w/ concurrent updates. (googleapis#3632)

d5e958c

* Also add systest for user exception aborting transaction.

landrito pushed a commit to landrito/google-cloud-python that referenced this pull request Aug 22, 2017

Add systests for read/query w/ concurrent updates. (googleapis#3632)

521de98

* Also add systest for user exception aborting transaction.

parthea pushed a commit that referenced this pull request Mar 9, 2026

Add systests for read/query w/ concurrent updates. (#3632)

c7e8010

* Also add systest for user exception aborting transaction.


		with session.batch() as batch:
		batch.delete(self.TABLE, self.ALL)

Conversation

tseaver commented Jul 19, 2017

Uh oh!

dhermes left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

tseaver commented Jul 19, 2017

Uh oh!

theacodes commented Jul 19, 2017

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

theacodes commented Jul 19, 2017

Uh oh!

This comment was marked as spam.

Uh oh!

dhermes commented Jul 19, 2017

Uh oh!

dhermes commented Jul 19, 2017

Uh oh!

tseaver commented Jul 19, 2017

Uh oh!

dhermes commented Jul 19, 2017

Uh oh!

tseaver commented Jul 19, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants