Error when clicking "more" and having exactly 10 books.

[johnshunfan]

Deploy part 2 - using datastore > visit /books > click "more" > Internal server error

[theacodes]

Can you send over any stacktrace? Does this occur locally?

On Mon, Oct 12, 2015 at 1:05 PM Shun Fan notifications@github.com wrote:

Deploy part 2 - using datastore > visit /books > click "more" > Internal
server error

—
Reply to this email directly or view it on GitHub
#8.

[johnshunfan]

Seems to happen locally as well.

127.0.0.1 - - [12/Oct/2015 13:18:27] "GET /books/?page_token=CkUKEwoFdGl0bGUSChoIdGhlb2RvcmUSKmoVc35iaWd0YWJsZS1zZmFuLXRlc3QxchELEgRCb29rGICAgIC+goMKDBgAIAA= HTTP/1.1" 500 -
Traceback (most recent call last):
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py", line 1836, in call
return self.wsgi_app(environ, start_response)
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functionsrule.endpoint
File "/Users/sfan/test/2-structured-data/bookshelf/crud.py", line 26, in list
books, next_page_token = get_model().list(cursor=token)
File "/Users/sfan/test/2-structured-data/bookshelf/model_datastore.py", line 57, in list
entities, more_results, cursor = it.next_page()
File "/Library/Python/2.7/site-packages/gcloud/datastore/query.py", line 391, in next_page
pb.start_cursor = base64.b64decode(start_cursor)
File "/Users/sfan/.homebrew/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/base64.py", line 76, in b64decode
# Transform this exception for consistency
TypeError: Incorrect padding

[theacodes]

Thanks! Will investigate.

On Mon, Oct 12, 2015 at 1:20 PM Shun Fan notifications@github.com wrote:

Seems to happen locally as well.

127.0.0.1 - - [12/Oct/2015 13:18:27] "GET
/books/?page_token=CkUKEwoFdGl0bGUSChoIdGhlb2RvcmUSKmoVc35iaWd0YWJsZS1zZmFuLXRlc3QxchELEgRCb29rGICAgIC+goMKDBgAIAA=
HTTP/1.1" 500 -
Traceback (most recent call last):
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py",
line 1836, in call
return self.wsgi_app(environ, start_response)
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py",
line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py",
line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py",
line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py",
line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py",
line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py",
line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/Users/sfan/.homebrew/lib/python2.7/site-packages/flask/app.py",
line 1461, in dispatch_request
return self.view_functionsrule.endpoint http://**req.view_args
File "/Users/sfan/test/2-structured-data/bookshelf/crud.py", line 26, in
list
books, next_page_token = get_model().list(cursor=token)
File "/Users/sfan/test/2-structured-data/bookshelf/model_datastore.py",
line 57, in list
entities, more_results, cursor = it.next_page()
File "/Library/Python/2.7/site-packages/gcloud/datastore/query.py", line
391, in next_page
pb.start_cursor = base64.b64decode(start_cursor)
File
"/Users/sfan/.homebrew/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/base64.py",
line 76, in b64decode

Transform this exception for consistency

TypeError: Incorrect padding

—
Reply to this email directly or view it on GitHub
#8 (comment)
.

[johnshunfan]

https://github.com/GoogleCloudPlatform/getting-started-python/blob/master/bookshelf/model_datastore.py#L56
I think it has to do with this line. You return a token if there are 10 books, but if there are exactly 10 books, it will try to load an additional page when there aren't any more books.
Seems like there should be the same issue in the cloudsql backend.

[theacodes]

@dhermes sigh, cursors aren't encoded with the urlsafe base64 encoding. Is there any reason for this? Should user apps just explicitly translate to urlsafe or do you feel like this should be fixed in gcloud-python?

[dhermes]

Here's your sign line:
https://github.com/GoogleCloudPlatform/gcloud-python/blob/7e999cfb0af59712a4106c51b63325b5b63addaa/gcloud/datastore/query.py#L424

[dhermes]

I don't have an opinion on urlsafe vs. regular b64. Maybe someone else does?

[theacodes]

I have a pretty strong opinion that we should use urlsafe, especially given the context that a cursor is very often passed around via URLs or forms.

Also, ndb's cursors objects that serialize to urlsafe b64, so pleaseeeeee? :D

[dhermes]

I'm super busy for the next 7-10 days. Care to send a PR (I can code review from my phone on public transit, can't write code though)?

[theacodes]

Absolutely. Thanks!

[dhermes]

👍 Thank YOU Jon

[theacodes]

Closing this, as it should be fixed upstream.