Document BINARY_OP_ opcodes? - Documentation - Discussions on Python.org
Document BINARY_OP_ opcodes? Documentation pamelafox (Pamela Fox) February 14, 2023, 5:58pm 1 Hey there, I just spent a while trying to distinguish the difference between BINARY_OP_ADAPTIVE and BINARY_OP_MULTIPLY_FLOAT . The dis docs only have BINARY_OP : I did find BINARY_OP_MULTIPLY_FLOAT in the CPython codebase, but I couldn’t find BINARY_OP_ADAPTIVE in the CPython codebase at all, so I presume its name is dynamically concatenated.
What do you think of documenting the various BINARY_OP_ opcodes? Are there too many? Similarly wondering about LOAD_CONST__LOAD_FAST and RESUME_QUICK . smontanaro (Skip Montanaro) February 14, 2023, 6:54pm 2 I wasn’t aware any opcodes were documented outside of the source code. That said, there seems to be an error in the BINARY_OP documentation, namely the introduction version. That opcode (ignoring the variants you mentioned) has been around for as long as I’ve been messing around with Python (1.0 timeframe).
It was probably part of the first version Guidon released. My guess is that one or both of the variants you mentioned were added in 3.11. I say go for it. I’m sure a PR would be welcome. That said, if the dust from major virtual machine rework hadn’t settled yet, you might well be playing a game of whack-a-mole for a while. guido (Guido van Rossum) February 15, 2023, 1:31am 3 Where did you search? In the 3.11 sources, BINARY_OP_ADAPTIVE (and many other _ADAPTIVE ) opcodes exist; in main (to become 3.12) they no longer exist.
All the opcodes you mention are “specialized” opcodes, and will never be seen in the co_code attribute of code objects when requested from Python code. They are inserted dynamically in Python/specialize.c. The numeric values of these are dynamically calculated in code that is intentionally hard to find (but the values are revealed in Include/opcode.h). You might find a little more background information about this topic in PEP 659 (Specializing Adaptive Interpreter).
Basically, adaptive opcodes are a stepping stone towards specialization. LOAD_CONST__LOAD_FAST and friends are “super-instructions” that replace a pair (here, LOAD_CONST + LOAD_FAST) but save a little of the dispatch effort. I’d be reluctant to document any of these, since there’s such a small audience for the documentation, and things are changing quickly in 3.12. I’m curious what led you to find these in the first place?
4 Likes guido (Guido van Rossum) February 15, 2023, 1:33am 4 Actually, what has been around through 3.10 was separate opcodes BINARY_OP_ADD, BINARY_OP_MULTIPLY, and so on. In 3.11 these were combined into BINARY_OP, with the oparg indicating which operation to perform. So I think the version mentioned in the docs is correct. CAM-Gerlach (C.A.M. Gerlach) February 15, 2023, 10:23am 5 Yup; the opcodes are all documented in the 3.10 docs , and only removed in 3.11, as we documented in the opcodes section of the What’s New in 3.11 .
So all appears to be in order here. smontanaro (Skip Montanaro) February 15, 2023, 10:29am 6 Doh, thanks for the gentle correction. LBYL would have been appropriate here. My synapses are wearing out. I ask forgiveness. 1 Like guido (Guido van Rossum) February 15, 2023, 4:58pm 7 I feel you – it’s the same for me. pamelafox (Pamela Fox) February 15, 2023, 10:01pm 8 Ah, good point, I only checked main for the opcode names, I didn’t check the 3.11 tree.
I discovered them as I made a little tool ( https://www.dis-this.com/ ) for running dis online with Pyodide, and I added 3.11 support this week with a toggle to try the specializing adaptive interpreter (adaptive=True). My tool hyperlinks the opcodes to the reference, and that’s when I realized the specializing opcodes weren’t in the reference. I realize my tool is very niche - it was inspired from how I would sometimes run dis in my Python class when students wanted to understand the difference between two syntaxes that achieved the same result.
I mostly use it now when I am just curious. It’s like seeing into the Matrix. Anyway, if it’s not appropriate to document those opcodes due to their high likelihood to change, that’s fine. I just wanted to make sure the lack of documentation was intentional. Thanks! 3 Likes guido (Guido van Rossum) February 15, 2023, 11:20pm 9 Yeah, it’s definitely intentional (though we can argue about it). Maybe your tool can link to the source for the specialized opcodes.
Pretty cool tool BTW. Feature request: select the Python version. 2 Likes pamelafox (Pamela Fox) February 20, 2023, 12:39am 10 Good idea! You can now change the Python version: https://www.dis-this.com/ I have to reload the whole page to change the version, since Pyodide leaks into the window globals and it’s too messy to load a new Pyodide version after. Also, notably, Pyodide isn’t designed to stably support multiple versions (see discussion ) - but hey, it works.
As suggested, I also made it so the links go out to ceval.c for the undocumented opcodes in 3.11. If you’re curious, here’s a list of undocumented opcodes: ['BINARY_OP_ADAPTIVE', 'BINARY_OP_ADD_FLOAT', 'BINARY_OP_ADD_INT', 'BINARY_OP_ADD_UNICODE', 'BINARY_OP_INPLACE_ADD_UNICODE', 'BINARY_OP_MULTIPLY_FLOAT', 'BINARY_OP_MULTIPLY_INT', 'BINARY_OP_SUBTRACT_FLOAT', 'BINARY_OP_SUBTRACT_INT', 'BINARY_SUBSCR_ADAPTIVE', 'BINARY_SUBSCR_DICT', 'BINARY_SUBSCR_GETITEM', 'BINARY_SUBSCR_LIST_INT', 'BINARY_SUBSCR_TUPLE_INT', 'CALL_ADAPTIVE', 'CALL_PY_EXACT_ARGS', 'CALL_PY_WITH_DEFAULTS', 'COMPARE_OP_ADAPTIVE', 'COMPARE_OP_FLOAT_JUMP', 'COMPARE_OP_INT_JUMP', 'COMPARE_OP_STR_JUMP', 'EXTENDED_ARG_QUICK', 'JUMP_BACKWARD_QUICK', 'LOAD_ATTR_ADAPTIVE', 'LOAD_ATTR_INSTANCE_VALUE', 'LOAD_ATTR_MODULE', 'LOAD_ATTR_SLOT', 'LOAD_ATTR_WITH_HINT', 'LOAD_CONST__LOAD_FAST', 'LOAD_FAST__LOAD_CONST', 'LOAD_FAST__LOAD_FAST', 'LOAD_GLOBAL_ADAPTIVE', 'LOAD_GLOBAL_BUILTIN', 'LOAD_GLOBAL_MODULE', 'LOAD_METHOD_ADAPTIVE', 'LOAD_METHOD_CLASS', 'LOAD_METHOD_MODULE', 'LOAD_METHOD_NO_DICT', 'LOAD_METHOD_WITH_DICT', 'LOAD_METHOD_WITH_VALUES', 'PRECALL_ADAPTIVE', 'PRECALL_BOUND_METHOD', 'PRECALL_BUILTIN_CLASS', 'PRECALL_BUILTIN_FAST_WITH_KEYWORDS', 'PRECALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS', 'PRECALL_NO_KW_BUILTIN_FAST', 'PRECALL_NO_KW_BUILTIN_O', 'PRECALL_NO_KW_ISINSTANCE', 'PRECALL_NO_KW_LEN', 'PRECALL_NO_KW_LIST_APPEND', 'PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST', 'PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS', 'PRECALL_NO_KW_METHOD_DESCRIPTOR_O', 'PRECALL_NO_KW_STR_1', 'PRECALL_NO_KW_TUPLE_1', 'PRECALL_NO_KW_TYPE_1', 'PRECALL_PYFUNC', 'RESUME_QUICK', 'STORE_ATTR_ADAPTIVE', 'STORE_ATTR_INSTANCE_VALUE', 'STORE_ATTR_SLOT', 'STORE_ATTR_WITH_HINT', 'STORE_FAST__LOAD_FAST', 'STORE_FAST__STORE_FAST', 'STORE_SUBSCR_ADAPTIVE', 'STORE_SUBSCR_DICT', 'STORE_SUBSCR_LIST_INT', 'UNPACK_SEQUENCE_ADAPTIVE', 'UNPACK_SEQUENCE_LIST', 'UNPACK_SEQUENCE_TUPLE', 'UNPACK_SEQUENCE_TWO_TUPLE', 'DO_TRACING', 'HAS_CONST(op)', 'NB_ADD', 'NB_AND', 'NB_FLOOR_DIVIDE', 'NB_LSHIFT', 'NB_MATRIX_MULTIPLY', 'NB_MULTIPLY', 'NB_REMAINDER', 'NB_OR', 'NB_POWER', 'NB_RSHIFT', 'NB_SUBTRACT', 'NB_TRUE_DIVIDE', 'NB_XOR', 'NB_INPLACE_ADD', 'NB_INPLACE_AND', 'NB_INPLACE_FLOOR_DIVIDE', 'NB_INPLACE_LSHIFT', 'NB_INPLACE_MATRIX_MULTIPLY', 'NB_INPLACE_MULTIPLY', 'NB_INPLACE_REMAINDER', 'NB_INPLACE_OR', 'NB_INPLACE_POWER', 'NB_INPLACE_RSHIFT', 'NB_INPLACE_SUBTRACT', 'NB_INPLACE_TRUE_DIVIDE', 'NB_INPLACE_XOR', 'HAS_ARG(op)', 'IS_ARTIFICIAL(op)'] I think at least one of those, IS_ARTIFICIAL , will never show up in a disassembly. guido (Guido van Rossum) February 20, 2023, 4:59am 11 Most of that looks expected.
How did you compute this data? It looks like you scanned opcode.h. HAS_ARG(op) also looks like a macro you accidentally picked up from there. Also, I had no idea you were using Pyodide! That’s amazing. Of course, now I am getting greedy and I want the nightly 3.12 build too (I don’t even know if we have that, the latest alpha would do) but I suggest that you don’t put any effort in that, there’s only literally a handful o
The rest of this page is on the live site.