Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
b833b83
Implement BytesIO.peek()
marcelm Jan 22, 2022
50a2cfb
📜🤖 Added by blurb_it.
blurb-it[bot] Apr 10, 2022
eaa7672
Document BytesIO.peek()
marcelm Nov 9, 2022
00457ae
Implement with the help of read_bytes()
marcelm Nov 9, 2022
882579d
Add to What’s New
marcelm Jul 8, 2023
c1eed72
versionadded: 3.12 -> 3.13
marcelm Jul 8, 2023
afc200c
Remove unused variable
marcelm Jul 8, 2023
79ab9a4
Test tell() after peek()
marcelm Sep 22, 2023
b493914
Update docs, factor out peek_bytes, semantics
marcelm Sep 22, 2023
2a1c85c
Update Misc/NEWS.d/next/Library/2022-04-10-20-10-59.bpo-46375.8j1ogZ.rst
marcelm Sep 28, 2023
d398717
Update Modules/_io/bytesio.c
marcelm Sep 28, 2023
26d1e81
Update Modules/_io/bytesio.c
marcelm Sep 28, 2023
9a19ff9
Use SemBr
marcelm Sep 28, 2023
9300ade
Update Doc/whatsnew/3.13.rst
marcelm Sep 28, 2023
d214089
Apply suggestions from code review
marcelm Sep 28, 2023
d6691b8
Use a context manager around memio in test_peek
marcelm Sep 28, 2023
3e51adb
Add more tests for tell() after peek()
marcelm Sep 28, 2023
3661b65
Document why size < 0 can happen
marcelm Sep 28, 2023
cd40d77
Update Modules/_io/bytesio.c
marcelm Sep 29, 2023
04372bd
Do not update pos if peek_bytes failed
marcelm Sep 29, 2023
6b9ae8c
Size can be negative after truncate or seek
marcelm Sep 29, 2023
f7406f6
Test with size<0 and size>len(buf)
marcelm Sep 29, 2023
d9528e2
Test peek() after write()
marcelm Sep 29, 2023
bc8134b
Document BufferedReader.peek and BytesIO.peek similarly
marcelm Sep 29, 2023
b6ffca8
Comment
marcelm Sep 29, 2023
5fe5645
Make it more explicit that size is ignored
marcelm Sep 29, 2023
4126a64
Return an empty bytes object for size=0
marcelm Oct 23, 2023
1ea40c2
Simplify
marcelm Oct 23, 2023
77e04d6
Test peek(3) and peek(5)
marcelm Oct 23, 2023
4d2f2dd
Run clinic.py
marcelm Apr 10, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions Doc/library/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -739,6 +739,15 @@ than raw I/O does.

Return :class:`bytes` containing the entire contents of the buffer.

.. method:: peek(size=1, /)

Return bytes from the current position onwards without advancing the position.
At least one byte of data is returned if not at EOF.
Return an empty :class:`bytes` object at EOF.
If the size argument is negative or larger than the number of available bytes,
a copy of the buffer from the current position until the end is returned.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can understand that size=-1 or size=None return the whole content. But I'm surprised that size=0 returns something different than an empty string or raise an exception.

I suggest to return an empty string when peek(0) is called, it would be similar to read(0).

Copy link
Copy Markdown
Author

@marcelm marcelm Oct 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, changed now.

This was originally for (perceived) consistency with BufferedReader.peek(), which does not return empty bytes objects for size=0. But then BufferedReader.peek() ignores the size anyway.


.. versionadded:: 3.15

.. method:: read1(size=-1, /)

Expand Down Expand Up @@ -772,8 +781,13 @@ than raw I/O does.

.. method:: peek(size=0, /)

Return bytes from the stream without advancing the position. The number of
bytes returned may be less or more than requested. If the underlying raw
Return bytes from the current position onwards without advancing the position.
At least one byte of data is returned if not at EOF.
Return an empty :class:`bytes` object at EOF.
At most one single read on the underlying raw stream is done to satisfy the call.
The *size* argument is ignored.
The number of read bytes depends on the buffer size and the current position in the internal buffer.
If the underlying raw
stream is non-blocking and the operation would block, returns empty bytes.

.. method:: read(size=-1, /)
Expand Down
8 changes: 8 additions & 0 deletions Doc/whatsnew/3.15.rst
Original file line number Diff line number Diff line change
Expand Up @@ -840,6 +840,14 @@ inspect
for :func:`~inspect.getdoc`.
(Contributed by Serhiy Storchaka in :gh:`132686`.)


io
--

* Add :meth:`io.BytesIO.peek`.
(Contributed by Marcel Martin in :gh:`90533`.)


json
----

Expand Down
7 changes: 7 additions & 0 deletions Lib/_pyio.py
Original file line number Diff line number Diff line change
Expand Up @@ -996,6 +996,13 @@ def tell(self):
raise ValueError("tell on closed file")
return self._pos

def peek(self, size=1):
if self.closed:
raise ValueError("peek on closed file")
if size < 0:
return self._buffer[self._pos:]
return self._buffer[self._pos:self._pos + size]

def truncate(self, pos=None):
if self.closed:
raise ValueError("truncate on closed file")
Expand Down
42 changes: 42 additions & 0 deletions Lib/test/test_io/test_memoryio.py
Original file line number Diff line number Diff line change
Expand Up @@ -566,6 +566,48 @@ def test_issue141311(self):
buf = bytearray(2)
self.assertEqual(0, memio.readinto(buf))

def test_peek(self):
buf = self.buftype("1234567890")
with self.ioclass(buf) as memio:
self.assertEqual(memio.tell(), 0)
self.assertEqual(memio.peek(1), buf[:1])
self.assertEqual(memio.peek(1), buf[:1])
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can add tests reading 3 and 5 bytes? It seems like most tests read 1 byte or everything.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely, added now.

self.assertEqual(memio.peek(), buf[:1])
self.assertEqual(memio.peek(3), buf[:3])
self.assertEqual(memio.peek(5), buf[:5])
self.assertEqual(memio.peek(0), b"")
self.assertEqual(memio.peek(len(buf) + 100), buf)
self.assertEqual(memio.peek(-1), buf)
self.assertEqual(memio.tell(), 0)
memio.read(1)
self.assertEqual(memio.tell(), 1)
self.assertEqual(memio.peek(1), buf[1:2])
self.assertEqual(memio.peek(), buf[1:2])
self.assertEqual(memio.peek(3), buf[1:4])
self.assertEqual(memio.peek(5), buf[1:6])
self.assertEqual(memio.peek(0), b"")
self.assertEqual(memio.peek(len(buf) + 100), buf[1:])
self.assertEqual(memio.peek(-1), buf[1:])
self.assertEqual(memio.tell(), 1)
memio.read()
self.assertEqual(memio.tell(), len(buf))
self.assertEqual(memio.peek(1), self.EOF)
self.assertEqual(memio.peek(3), self.EOF)
self.assertEqual(memio.peek(5), self.EOF)
self.assertEqual(memio.peek(0), b"")
self.assertEqual(memio.tell(), len(buf))
# Peeking works after writing
abc = self.buftype("abc")
memio.write(abc)
self.assertEqual(memio.peek(), self.EOF)
memio.seek(len(buf))
self.assertEqual(memio.peek(), abc[:1])
self.assertEqual(memio.peek(-1), abc)
self.assertEqual(memio.peek(len(abc) + 100), abc)
self.assertEqual(memio.tell(), len(buf))

self.assertRaises(ValueError, memio.peek)

def test_unicode(self):
memio = self.ioclass()

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add :meth:`io.BytesIO.peek`.
48 changes: 45 additions & 3 deletions Modules/_io/bytesio.c
Original file line number Diff line number Diff line change
Expand Up @@ -420,8 +420,9 @@ _io_BytesIO_tell_impl(bytesio *self)
return PyLong_FromSsize_t(self->pos);
}

// Read without advancing position
static PyObject *
read_bytes_lock_held(bytesio *self, Py_ssize_t size)
peek_bytes_lock_held(bytesio *self, Py_ssize_t size)
{
_Py_CRITICAL_SECTION_ASSERT_OBJECT_LOCKED(self);

Expand All @@ -432,7 +433,6 @@ read_bytes_lock_held(bytesio *self, Py_ssize_t size)
if (size > 1 &&
self->pos == 0 && size == PyBytes_GET_SIZE(self->buf) &&
FT_ATOMIC_LOAD_SSIZE_RELAXED(self->exports) == 0) {
self->pos += size;
return Py_NewRef(self->buf);
}

Expand All @@ -444,10 +444,19 @@ read_bytes_lock_held(bytesio *self, Py_ssize_t size)
}

output = PyBytes_AS_STRING(self->buf) + self->pos;
self->pos += size;
return PyBytes_FromStringAndSize(output, size);
}

static PyObject *
read_bytes_lock_held(bytesio *self, Py_ssize_t size)
{
PyObject *bytes = peek_bytes_lock_held(self, size);
if (bytes != NULL) {
self->pos += size;
}
return bytes;
}

/*[clinic input]
@critical_section
_io.BytesIO.read
Expand Down Expand Up @@ -499,6 +508,38 @@ _io_BytesIO_read1_impl(bytesio *self, Py_ssize_t size)
return _io_BytesIO_read_impl(self, size);
}


/*[clinic input]
_io.BytesIO.peek
size: Py_ssize_t = 1
/

Return bytes from the stream without advancing the position.

If the size argument is negative, read until EOF is reached.
Return an empty bytes object at EOF.
[clinic start generated code]*/

static PyObject *
_io_BytesIO_peek_impl(bytesio *self, Py_ssize_t size)
/*[clinic end generated code: output=fa4d8ce28b35db9b input=1510f0fcf77c0048]*/
{
CHECK_CLOSED(self);

/* adjust invalid sizes */
Py_ssize_t n = self->string_size - self->pos;
if (size < 0 || size > n) {
size = n;
/* n can be negative after truncate() or seek() */
if (size < 0) {
size = 0;
}
}
return peek_bytes_lock_held(self, size);
}



/*[clinic input]
@critical_section
_io.BytesIO.readline
Expand Down Expand Up @@ -1135,6 +1176,7 @@ static struct PyMethodDef bytesio_methods[] = {
_IO_BYTESIO_READLINE_METHODDEF
_IO_BYTESIO_READLINES_METHODDEF
_IO_BYTESIO_READ_METHODDEF
_IO_BYTESIO_PEEK_METHODDEF
_IO_BYTESIO_GETBUFFER_METHODDEF
_IO_BYTESIO_GETVALUE_METHODDEF
_IO_BYTESIO_SEEK_METHODDEF
Expand Down
48 changes: 47 additions & 1 deletion Modules/_io/clinic/bytesio.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading