|
1 | 1 | Introduction |
2 | 2 | ============ |
3 | 3 |
|
4 | | -Mypyc is a compiler for a strict, statically typed Python variant that |
5 | | -generates CPython C extension modules. Code compiled with mypyc is |
6 | | -often much faster than CPython. Mypyc uses Python `type hints |
| 4 | +Mypyc compiles Python modules to C extensions. It uses standard Python |
| 5 | +`type hints |
7 | 6 | <https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html>`_ to |
8 | | -generate fast code, and it also restricts the use of some dynamic |
9 | | -Python features to gain performance. |
| 7 | +generate fast code. |
| 8 | + |
| 9 | +The compiled language is a strict, statically typed Python variant. |
| 10 | +It restricts the use of some dynamic Python features to gain performance, |
| 11 | +but it's mostly compatible with standard Python. |
10 | 12 |
|
11 | 13 | Mypyc uses `mypy <http://www.mypy-lang.org/>`_ to perform type |
12 | | -checking and type inference. Most type checking features in the stdlib |
| 14 | +checking and type inference. Most type system features in the stdlib |
13 | 15 | `typing <https://docs.python.org/3/library/typing.html>`_ module are |
14 | | -supported, including generic types, optional and union types, tuple |
15 | | -types, and type variables. Using type hints is not necessary, but type |
16 | | -annotations are the key to impressive performance gains. |
| 16 | +supported. |
17 | 17 |
|
18 | | -Compiled modules can import arbitrary Python modules, including |
19 | | -third-party libraries, and compiled modules can be freely used from |
20 | | -other Python modules. Often you'd use mypyc to only compile modules |
21 | | -with performance bottlenecks. |
| 18 | +Compiled modules can import arbitrary Python modules and third-party |
| 19 | +libraries. |
22 | 20 |
|
23 | 21 | You can run the modules you compile also as normal, interpreted Python |
24 | | -modules. Mypyc only compiles valid Python code. This means that all |
25 | | -Python developer tools and debuggers can be used, though some only |
26 | | -fully work in interpreted mode. |
| 22 | +modules. |
27 | 23 |
|
28 | | -How fast is mypyc |
29 | | ------------------ |
| 24 | +You can roughly expect speedups like these (2x improvement means half the |
| 25 | +runtime): |
30 | 26 |
|
31 | | -The speed improvement from compilation depends on many factors. |
32 | | -Certain operations will be a lot faster, while others will get no |
33 | | -speedup. |
| 27 | +* Existing code with type annotations often gets **1.5x to 5x** faster. |
34 | 28 |
|
35 | | -These estimates give a rough idea of what to expect (2x improvement |
36 | | -halves the runtime): |
| 29 | +* Code tuned for mypyc can be **5x to 15x** faster. |
37 | 30 |
|
38 | | -* Typical code with type annotations may get **1.5x to 5x** faster. |
| 31 | +There is no simple answer to how fast your code will be when compiled. |
| 32 | +You should try it out! |
39 | 33 |
|
40 | | -* Typical code with *no* type annotations may get **1.0x to 1.3x** |
41 | | - faster. |
| 34 | +Mypyc currently aims to speed up non-numeric code, such as server |
| 35 | +applications. We also use mypyc to compile mypyc, of course! |
42 | 36 |
|
43 | | -* Code optimized for mypyc may get **5x to 10x** faster. |
| 37 | +Motivation |
| 38 | +---------- |
44 | 39 |
|
45 | | -Remember that only performance of compiled modules improves. Time |
46 | | -spent in libraries or on I/O will not change (unless you also compile |
47 | | -libraries). |
| 40 | +Though Python has been successful without a good performance story |
| 41 | +for non-numeric code, speed still matters: |
48 | 42 |
|
49 | | -Why speed matters |
50 | | ------------------ |
| 43 | +1. Users prefer more efficient and responsive software and libraries. |
51 | 44 |
|
52 | | -Faster code has many benefits, some obvious and others less so: |
| 45 | +2. You need less hardware to run your server application and save money. |
53 | 46 |
|
54 | | -* Users prefer efficient and responsive applications, tools and |
55 | | - libraries. |
| 47 | +3. You'll waste less time waiting for your tests and jobs to finish. |
56 | 48 |
|
57 | | -* If your server application is faster, you need less hardware, which |
58 | | - saves money. |
| 49 | +4. Faster code correlates with less energy use and is better for the |
| 50 | + environment. |
59 | 51 |
|
60 | | -* Faster code uses less energy, especially on servers that run 24/7. |
61 | | - This lowers your environmental footprint. |
| 52 | +Perks |
| 53 | +----- |
62 | 54 |
|
63 | | -* If tests or batch jobs run faster, you'll be more productive and |
64 | | - save time. |
| 55 | +**Easy to get started.** Compiled code looks and behaves like normal |
| 56 | +Python code. You get the benefits of static typing while using the |
| 57 | +syntax, libraries and idioms you (and millions of developers) already |
| 58 | +know. |
65 | 59 |
|
66 | | -How does mypyc work |
67 | | -------------------- |
| 60 | +**Expressive types.** Mypyc fully supports standard Python type hints. |
| 61 | +Mypyc has local type inference, generics, optional types, tuple types, |
| 62 | +union types, and more. Type hints act as machine-checked |
| 63 | +documentation, making code not only faster but also easier to |
| 64 | +understand and modify. |
68 | 65 |
|
69 | | -Mypyc produces fast code via several techniques: |
| 66 | +**Fast program startup.** Mypyc uses ahead-of-time compilation, so |
| 67 | +compilation does not slow down program startup. Slow program startup |
| 68 | +is a common problem with JIT compilers. |
70 | 69 |
|
71 | | -* Mypyc uses *ahead-of-time compilation* to native code. This removes |
72 | | - CPython interpreter overhead. |
73 | | - |
74 | | -* Mypyc enforces type annotations (and type comments) at runtime, |
75 | | - raising ``TypeError`` if runtime types don't match annotations. This |
76 | | - lets mypyc use operations specialized to specific types. |
| 70 | +**Python ecosystem supported.** Mypyc runs on top of CPython, the |
| 71 | +standard Python implementation. Code can freely use standard library |
| 72 | +features. Use pip to install any third-party libraries you need, |
| 73 | +including C extensions. |
77 | 74 |
|
78 | | -* Mypyc uses *early binding* to resolve called functions and other |
79 | | - references at compile time. Mypyc avoids many namespace dictionary |
80 | | - lookups. |
| 75 | +**Migration path for existing Python code.** Existing Python code |
| 76 | +often requires only minor changes to compile using mypyc. |
81 | 77 |
|
82 | | -* Mypyc assumes that most compiled functions, compiled classes, and |
83 | | - attributes declared ``Final`` are immutable (and tries to enforce |
84 | | - this). |
| 78 | +**Waiting for compilation is optional.** Compiled code also runs as |
| 79 | +normal Python code. You can use interpreted Python during development, |
| 80 | +with familiar and fast workflows. |
85 | 81 |
|
86 | | -* Most classes are compiled to *C extension classes*. They use |
87 | | - `vtables <https://en.wikipedia.org/wiki/Virtual_method_table>`_ for |
88 | | - fast method calls and attribute access. |
| 82 | +**Runtime type safety.** Mypyc protects you from segfaults and memory |
| 83 | +corruption. Any unexpected runtime type safety violation is a bug in |
| 84 | +mypyc. |
89 | 85 |
|
90 | | -* Mypyc uses efficient (unboxed) representations for some primitive |
91 | | - types, such as integers and booleans. |
| 86 | +**Find errors statically.** Mypyc uses mypy for static type checking |
| 87 | +that will catch many bugs. This saves time you'd otherwise spend |
| 88 | +debugging. |
92 | 89 |
|
93 | | -Why mypyc |
| 90 | +Use cases |
94 | 91 | --------- |
95 | 92 |
|
96 | | -Here are some mypyc properties and features that can be useful. |
| 93 | +**Fix performance bottlenecks.** Often most time is spent in a few |
| 94 | +Python modules or functions. Add type annotations and compile these |
| 95 | +modules for easy performance gains. |
97 | 96 |
|
98 | | -**Powerful Python types.** Mypyc leverages most features of standard |
99 | | -Python type hint syntax, unlike tools such as Cython, which focus on |
100 | | -lower-level types. Our aim is that writing code feels natural and |
101 | | -Pythonic. Mypyc supports a modern type system with powerful features |
102 | | -such as local type inference, generics, optional types, tuple types |
103 | | -and union types. Type hints act as machine-checked documentation, |
104 | | -making code easier to understand and modify. |
| 97 | +**Compile it all.** During development you use interpreted mode, for a |
| 98 | +quick edit-run cycle. In releases all non-test code is compiled. This |
| 99 | +is how mypy got a *4x performance improvement* over interpreted Python. |
105 | 100 |
|
106 | | -**Fast program startup.** Python implementations using a JIT compiler, |
107 | | -such as PyPy, slow down program startup, sometimes significantly. |
108 | | -Mypyc uses ahead-of-time compilation, so compilation does not slow |
109 | | -down program startup. |
| 101 | +**Take advantage of existing type hints.** If you already use type |
| 102 | +annotations in your code, adopting mypyc will be easier. You've already |
| 103 | +done most of the work needed to use mypyc. |
110 | 104 |
|
111 | | -**Python ecosystem compatibility.** Since mypyc uses the standard |
112 | | -CPython runtime, you can freely use the stdlib and use pip to install |
113 | | -arbitary third-party libraries, including C extensions. |
| 105 | +**Alternative to a lower-level language.** Instead of writing |
| 106 | +performance-critical code in C, C++, Cython or Rust, you may get good |
| 107 | +performance while staying in the comfort of Python. |
114 | 108 |
|
115 | | -**Migration path for existing Python code.** Existing Python code |
116 | | -often requires only minor changes to compile using mypyc. |
117 | | - |
118 | | -**No need to wait for compilation.** Compiled code also runs as normal |
119 | | -Python code. You can use interpreted Python during development, with |
120 | | -familiar workflows. |
| 109 | +**Migrate C extensions.** Maintaining C extensions is not always fun |
| 110 | +for a Python developer. With mypyc you may get performance similar to |
| 111 | +the original C, with the convenience of Python. |
121 | 112 |
|
122 | | -**Runtime type safety.** Mypyc aims to protect you from segfaults and |
123 | | -memory corruption. We consider any unexpected runtime type safety |
124 | | -violation as a bug. |
125 | | - |
126 | | -**Find errors statically.** Mypyc uses mypy for powerful static type |
127 | | -checking that will catch many bugs, saving you from a lot of |
128 | | -debugging. |
| 113 | +How does it work |
| 114 | +---------------- |
129 | 115 |
|
130 | | -**Easy path to static typing.** Mypyc lets Python developers easily |
131 | | -dip their toes into modern static typing, without having to learn all |
132 | | -new syntax, libraries and idioms. |
| 116 | +Mypyc uses several techniques to produce fast code: |
133 | 117 |
|
134 | | -Use cases for mypyc |
135 | | -------------------- |
| 118 | +* Mypyc uses *ahead-of-time compilation* to native code. This removes |
| 119 | + CPython interpreter overhead. |
136 | 120 |
|
137 | | -Here are examples of use cases where mypyc can be effective. |
| 121 | +* Mypyc enforces type annotations (and type comments) at runtime, |
| 122 | + raising ``TypeError`` if runtime values don't match annotations. |
138 | 123 |
|
139 | | -**Address a performance bottleneck.** Profiling shows that most time |
140 | | -is spent in a certain Python module. Add type annotations and compile |
141 | | -the module for performance gains. |
| 124 | +* Compiled code uses optimized, type-specific primitives. |
142 | 125 |
|
143 | | -**Leverage existing type hints.** You already use mypy to type check |
144 | | -your code. Using mypyc will now be easy, since you already use static |
145 | | -typing. |
| 126 | +* Mypyc uses *early binding* to resolve called functions and name |
| 127 | + references at compile time. Mypyc avoids many dynamic namespace |
| 128 | + lookups. |
146 | 129 |
|
147 | | -**Compile everything.** You want your whole application to be fast. |
148 | | -During development you use interpreted mode, for a quick edit-run |
149 | | -cycle, but in your releases all (non-test) code is compiled. This is |
150 | | -how mypy achieved a 4x performance improvement using mypyc. |
| 130 | +* Classes are compiled to *C extension classes*. They use `vtables |
| 131 | + <https://en.wikipedia.org/wiki/Virtual_method_table>`_ for fast |
| 132 | + method calls and attribute access. |
151 | 133 |
|
152 | | -**Alternative to C.** You are writing a new module that must be fast. |
153 | | -You write the module in Python, and try to use operations that mypyc |
154 | | -can optimize well. The module is much faster when compiled, and you've |
155 | | -saved a lot of effort compared to writing an extension in C (and you |
156 | | -don't need to know C). |
| 134 | +* Mypyc treats compiled functions, classes, and attributes declared |
| 135 | + ``Final`` as immutable. |
157 | 136 |
|
158 | | -**Rewriting a C extension.** You've written a C extension, but |
159 | | -maintaining C code is no fun. You might be able to switch to Python |
160 | | -and use mypyc to get performance comparable to the original C. |
| 137 | +* Mypyc has memory-efficient, unboxed representions for integers and |
| 138 | + booleans. |
161 | 139 |
|
162 | 140 | Development status |
163 | 141 | ------------------ |
164 | 142 |
|
165 | 143 | Mypyc is currently *alpha software*. It's only recommended for |
166 | | -production use cases if you are willing to contribute fixes or to work |
167 | | -around issues you will encounter. |
| 144 | +production use cases with careful testing, and if you are willing to |
| 145 | +contribute fixes or to work around issues you will encounter. |
0 commit comments