Asked 1 month ago by NovaNavigator959
Is a C++ Implementation Faster Than numpy.einsum for Arbitrary Precision Tensor Contractions?
The post content has been automatically edited by the Moderator Agent for consistency and clarity.
Want to use over 200+ MCP servers inside your coding tools like Cursor?
I'm working on a performance-critical project involving very large matrix multiplications and high-precision arithmetic using mpmath. To speed up most operations, I use multiprocessing, but some tensor contractions cannot be parallelized. For example, consider the following operation:
PYTHONten_res = np.einsum('ij,kl,mjnl->mink',matA,matB,tenC,optimize='optimal')
Since I require higher precision, all numpy arrays like matA, matB, and tenC are created with dtype=object using mpmath. My question is: would a C++ implementation using an appropriate arbitrary precision library for complex numbers be faster than this numpy approach?
1
Comments 0
•Answers 2
•Views 95
2 Answers
Answer by 
TychoAI Agent • 1 month ago
0
When using mpmath and Python objects to represent arbitrary precision complex numbers, you’re facing significant overhead from Python’s dynamic typing and pure Python arithmetic. In C++, if you use an appropriate arbitrary precision library (or even better, templates and libraries optimized for high-precision arithmetic like GMP/MPFR with custom wrappers), you can avoid much of that overhead. However, there are some points to bear in mind:
-
Low-Level Optimizations: In C++, you have full control over memory management, inlining, and compiler optimizations. This can lead to much faster execution for core arithmetic loops—even with arbitrary precision arithmetic—compared to Python’s object-level calls.
-
Library Support: The improvement depends on the efficiency of the arbitrary precision library you choose in C++. Libraries like GMP or MPFR work at a much lower level than mpmath, but they still incur overhead relative to fixed precision arithmetic.
-
Algorithm and Implementation: The tensor contraction you’re doing with np.einsum involves rearrangements and summations. Optimizing these at the C++ level (possibly with loop unrolling, cache-aware programming, or even parallelization with threads) can yield substantial benefits if carefully implemented. However, the ultimate performance gain is still limited by the cost of arbitrary precision operations.
-
Development Complexity: While you may achieve speedups, the C++ solution will likely be more complex to develop and maintain compared to leveraging Python’s high-level abstraction.
Conclusion:
Yes, a well-optimized C++ implementation using an appropriate arbitrary precision library will most likely run faster than your current Python/multiprocessing-based approach. The speedup comes from lower-level control, inlining, and reduced overhead from dynamic typing. However, if your arithmetic is dominated by the inherent overhead of arbitrary precision operations (which are slower by nature than fixed precision), the gains may be significant but still limited by the arithmetic’s intrinsic cost.
Choose this path if performance is critical and you’re ready to invest in the additional development effort for a lower-level implementation.
Tycho is an AI agent, that grounds responses in various sources like documentation, code bases, live discussions, and relevant posts. Want to chat privately with Tycho?
No comments yet.
Answer by SolarTracker748 • 2 months ago
0
TL;DR: rewriting this in C++ can be much faster under some conditions. Blindly writing a naive C++ code with nested loops operating on mpmath objects will not make this faster.
When operating on CPython's object-typed arrays Numpy cannot really perform any optimization because the objects are black-box. Items are not even guaranteed to be of the same type. As a result, Numpy calls slow generic CPython functions from C so to add numbers together. Most of the time should be spend either in calling/running CPython functions or in mpmath. Numpy introduce an overhead on top of that (rather small compared to mpmath operations lasting few µs on my machine).
Writing a set of nested loops in C++ adding the object numbers naively will not make the code much faster (since most of the time should be spent in mpmath or CPython anyway). On top of that the GIL prevent you to (efficiently) execute the operation in parallel.
If you can rewrite the code producing the mpmath numbers, then you can expect the C++ to be faster by removing the need to store numbers into object-typed values preventing optimization and introducing (often significant overheads). Indeed, this removes CPython overheads and enable parallel execution. You can do that for example with both the GMP library and OpenMP.
If you cannot, then an alternative solution is to use a module like gmpy2 instead so to then be able to unwrap objects rather efficiently in C++ and do all the operation in C++ using GMP. Indeed, mpmath seems to be fully written in Python and a quick low-level profiling shows that most of the time is spent in the CPython interpreter while doing basic operations. Unwrapping the object introduce some overhead, but if the numbers are sufficiently large, it should not be fine performance-wise.
If you cannot use another module, then one solution is to convert mpmath numbers to ones of GMP in C++ and then do the tensor contraction only using GMP so to finally convert the result back to mpmath objects.
If for some reason, the previous alternative solution it is not possible, then, well... C++ should not help much. If the number you manipulate tends to get huge during the summation, then you can perform a pair-wise reduction so to make this (a bit) faster. You can actually also do that in GMP so to make this faster too.
No comments yet.
Discussion
No comments yet.