Add a new `ApiMisuseException` - Core Development - Discussions on Python.org
Add a new `ApiMisuseException` Core Development markshannon (Mark Shannon) January 30, 2025, 9:42am 1 We currently raise SystemError for a variety of API misuses. This is misleading to users and results in spurious bug reports to CPython. I think we should add an ApiMisuseException for this, and keep SystemError for actual system errors. Some examples where ApiMisuseException would replace SystemError : A builtin function returns non-NULL and sets the exception or returns NULL and does not set the exception.
PyBytes_FromStringAndSize , PyUnicode_FromStringAndSize , etc. is called with a negative size PyFile_WriteString is called with a NULL file PyUnicode_New is called with an invalid character One of the PyArg_Parse variants is passed an invalid formats or arguments There are, no doubt, other examples. 7 Likes davidhewitt (David Hewitt) January 30, 2025, 9:56am 2 This would be nice for downstream projects like PyO3, where we could likely assume that any such ApiMisuseException in packages implemented in Rust is indicative of a bug in PyO3 1 Like mgorny (Michał Górny) January 30, 2025, 10:24am 3 I dare say that’s what normally qualifies as an “assertion error”, but I suppose it makes sense to keep it separate from explicit user assertions. encukou (Petr Viktorin) January 30, 2025, 10:47am 4 +1, but to bikeshed…: which API was misused?
Perhaps name it BadInternalCall (after the function ). Perhaps slightly incorrect (it’s not always a call ), but more obviously related to the (interpreter) internals. 5 Likes philthompson (Phil Thompson) January 30, 2025, 11:25am 5 What about 3rd party extension modules (arguably a more likely source of bad API calls)? 1 Like da-woods (Da Woods) January 30, 2025, 11:43am 6 I think that’s the point - it would mostly be third-party extensions that trigger this exception.
And Mark’s (probably optimistic) hope its that people would report the bugs to the authors of those extensions rather than to the authors of Python. storchaka (Serhiy Storchaka) January 30, 2025, 2:06pm 7 What are examples of SystemError which are not API misuses? Would not be easier and more compatibly to add InternalSystemError for such cases if there are any? 3 Likes ZeroIntensity (Peter Bierma) January 30, 2025, 2:25pm 8 Re: The `SystemError` description is misleading · Issue #129407 · python/cpython · GitHub I think it makes sense to add a separate exception, but I’m a bit concerned about what weird things we might break by changing exception types ( e.g. , will tests start failing because they were expecting SystemError ?) Another interesting case to think about is extensions that themselves expose ABIs; in the past, I’ve raised SystemError from pyawaitable to denote misuse, but in practice, that caused some confusion.
It would be quite helpful if a new exception could be used for extension ABIs instead of SystemError . Adding on the bikeshedding: I like something along the lines of NativeError more than BadInternalCall –IMO, we should preferably keep the phrase “internal” out of the exception if it wasn’t caused by the core. steve.dower (Steve Dower) January 30, 2025, 2:45pm 9 My initial thinking is that this is the more compatible way to do it.
Any SystemError that’s entirely internal can be a subclass of SystemError , which can be distinguished by handlers that care but still be handled by those that don’t. But I suspect we don’t actually have any internal ones, as we tend to rely on plain old assertions for cases where we are sure we control the inputs. So perhaps we just need to document it more clearly? gpshead (Gregory P. Smith) January 30, 2025, 11:05pm 10 If we had a way to often detect if the likely cause of the error (just the C caller?) is “internals” vs “non-internals” code we could even differentiate by subtypes… being correctish even 90% of the time here is probably useful? brettcannon (Brett Cannon) January 30, 2025, 11:07pm 11 FYI this doesn’t follow PEP 8 as it says to capitalize acronyms .
3 Likes storchaka (Serhiy Storchaka) January 31, 2025, 8:25am 12 Actually, I encountered one case of an internal SystemError 2 years ago. SystemError in re.match with a "*+" pattern · Issue #101955 · python/cpython · GitHub PyErr_SetString(PyExc_SystemError, "The span of capturing group is wrong," " please report a bug for the re module."); It clearly states that it is is an internal error. But I don’t think that there are many such cases.
The internals are mostly bug-free, and consistency checks are usually implemented as asserts. Even in case of genuine internal errors, it is not always possible to differentiate them from API misuse, because the internal code uses the API, so an internal bug can lead to API misuse. 2 Likes