Repository navigation
Accept file objects in MODE_AUTO and check the database argument #471
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
d64a469
d7041d3
abe833f
896fb5b
54de57b
b8ca09a
32da20a
54afbd2
f996560
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,7 +15,7 @@ | |
| Mode, | ||
| ) | ||
| from .errors import InvalidDatabaseError | ||
| from .reader import Reader | ||
| from .reader import _PATH_TYPES, Reader, _is_descriptor, _type_name | ||
|
|
||
| if TYPE_CHECKING: | ||
| from .types import DatabaseSource | ||
|
|
@@ -48,16 +48,27 @@ def open_database( | |
|
|
||
| Arguments: | ||
| database: A path to a valid MaxMind DB file such as a GeoIP database | ||
| file, or a file descriptor in the case of MODE_FD. | ||
| file, or a binary file object for MODE_FD or MODE_AUTO. | ||
| MODE_MMAP, MODE_FILE and MODE_MEMORY also accept the file | ||
| descriptor of a regular file. MODE_MEMORY reads it from its | ||
| current offset, the others from the start. Without the C | ||
| extension, MODE_AUTO accepts one too. Once the reader opens | ||
| a file descriptor, it closes it, even when the file is not | ||
| a valid database. If an error comes first, for example for | ||
| a mode that takes no descriptor or for a directory, the | ||
| caller still owns it. The reader does not close a file | ||
| object, but it closes a buffer that read() returns if the | ||
| buffer has close(), such as an mmap. | ||
| mode: mode to open the database with. Valid mode are: | ||
| * MODE_MMAP_EXT - use the C extension with memory map. | ||
| * MODE_MMAP - read from memory map. Pure Python. | ||
| * MODE_FILE - read database as standard file. Pure Python. | ||
| * MODE_MEMORY - load database into memory. Pure Python. | ||
| * MODE_FD - the param passed via database is a file descriptor, not | ||
| a path. This mode implies MODE_MEMORY. | ||
| * MODE_FD - the param passed via database is a binary file | ||
| object, not a path. This mode implies MODE_MEMORY. | ||
| * MODE_AUTO - tries MODE_MMAP_EXT, MODE_MMAP, MODE_FILE in that | ||
| order. Default mode. | ||
| order. Reads a file object into memory, as MODE_FD | ||
| does. Default mode. | ||
|
|
||
| """ | ||
| if mode not in ( | ||
|
|
@@ -72,23 +83,31 @@ def open_database( | |
| raise ValueError(msg) | ||
|
|
||
| has_extension = _extension and hasattr(_extension, "Reader") | ||
| use_extension = has_extension if mode == MODE_AUTO else mode == MODE_MMAP_EXT | ||
|
|
||
| if not use_extension: | ||
| return Reader(database, mode) | ||
|
|
||
| if not has_extension: | ||
| if mode == MODE_MMAP_EXT and not has_extension: | ||
| msg = "MODE_MMAP_EXT requires the maxminddb.extension module to be available" | ||
| raise ValueError( | ||
| msg, | ||
| ) | ||
|
|
||
| # The C type exposes the same API as the Python Reader, so for type | ||
| # checking purposes, pretend it is one. (Ideally this would be a subclass | ||
| # of, or share a common parent class with, the Python Reader | ||
| # implementation.) The extension accepts only a path. It raises TypeError | ||
| # for a file descriptor or a file object. | ||
| return cast("Reader", _extension.Reader(database, mode)) # type: ignore[arg-type] | ||
| # The extension accepts only a path, so MODE_AUTO gives a file object to | ||
| # the pure Python reader. It still refuses a file descriptor, as before. | ||
| # The cast pretends the C reader is the pure Python Reader, which has the | ||
| # same API. | ||
| if mode in (MODE_AUTO, MODE_MMAP_EXT) and has_extension: | ||
| if isinstance(database, _PATH_TYPES): | ||
| return cast("Reader", _extension.Reader(database, mode)) | ||
| # Leave a bool to the pure Python reader, which refuses it. | ||
| is_descriptor = _is_descriptor(database) | ||
| if mode == MODE_MMAP_EXT or is_descriptor: | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In
So code that works in one environment fails (and leaks fds) in the other. This PR adds new code that keeps this split. Can we send a descriptor in 馃 Comment by Claude Opus 5.5.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agent reply on behalf of @oschwald. Not changed. 3.2.0 already had both behaviors. With the extension,
The docstrings describe both cases. |
||
| msg = f"The C extension requires a path ({_type_name(database)} given)." | ||
| if is_descriptor: | ||
| msg += " Use MODE_MMAP for a file descriptor." | ||
| elif callable(getattr(database, "read", None)): | ||
| msg += " Use MODE_FD for a file object." | ||
| raise TypeError(msg) | ||
|
|
||
| return Reader(database, mode) | ||
|
|
||
|
|
||
| __version__ = version("maxminddb") | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code checks the database type in two places.
open_databaseandReader._load_buffereach testisinstance(database, _PATH_TYPES),_is_descriptor()andcallable(getattr(database, "read", None)), and each builds its own hints.The messages already differ: one says "(X given). Use MODE_FD for a file object." and the other says "Unsupported database type (X). ...". One shared helper that returns the kind (path, descriptor, file object or other) would remove the repeated checks and keep the messages the same.
馃 Comment by Claude Opus 5.5.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agent reply on behalf of @oschwald.
Not changed.
_is_descriptoris the one shared check, and both places use it. Each place sorts the argument for its own reader. The extension takes only a path, but the pure Python reader also takes a descriptor or a file object. So each one gives the hint that fits its own reader.