Memestra!
Static code analysis for the detection of deprecated API calls in Python
Modern IDE features such as interactive debugging, linting, code formatting, and refactoring tools are now available in JupyterLab, thanks to the JupyterLab-LSP extension and the recent release of the Jupyterlab Visual Debugger.
A useful tool for developers is the ability to check for the use of deprecated APIs. Unfortunately, deprecation in Python is typically signaled using the standard DeprecationWarning, a technique that does not lend itself to static code analysis, leaving the discovery of deprecation warnings to the runtime. Others make use of decorators, such as the Deprecated library, which has the advantage of being much more compatible with static analysis.
A robust stack for Python static code analysis
We are indebted to the work of Serge Guelton, who is mostly known in the PyData community for the development of Pythran, which is an “ahead of time compiler for a subset of the Python language, with a focus on scientific computing”.
Most interestingly for us, Pythran is built upon a solid stack by the same author for Python static code analysis
- GAST: a generic AST to represent Python2 and Python3’s Abstract Syntax Tree(AST). GAST is a very popular project and a dependency of Tensorflow!
- Beniget: a collection of compile-time analysis tools for the Python Abstract Syntax Tree (AST). It is meant as a building block for Python compilers and static analyzers. More specifically, Beniget produces def-use and use-def chains, to map each node to the list of possible definitions of that node, and each node to the list of places where it is used.
Memestra!
Today, we are thrilled to announce the release of Memestra, a static analysis tool for Python, which detects the use of deprecated APIs. Memestra comes along with the pyls-memestra plugin for the Python Language Server.
PYLS-Memestra counts on Memestra and the JupyterLab-LSP to show deprecations that were flagged as deprecated in your code.
On top of the detection of the use of deprecated decorators, memestra provides a caching mechanism to speed up analysis and prevent the re-parsing of the code in use. Most interestingly, memestra cache files can also be produced with other tools, so that libraries that signal deprecations differently than with a decorator can still produce a memestra-compatible cache file.
Future Developments
The GAST and Beniget tools proved extremely robust and could be used for this new use case with almost no modification. We think that more features taking advantage of such static analysis could be brought to JupyterLab, such as taint analysis, better refactoring tools, and more.
Try it online!
You can try the stable version without the need of installing anything on your computer just by clicking on the image below:
Installation
Memestra and PYLS-Memestra are available on conda-forge and you can get it with mamba:
mamba install -c conda-forge pyls-lmemestraIt is also available on PyPI:
pip install pyls-memestraThe JupyterLab-LSP plugin is installed separately:
mamba install -c conda-forge nodejs jupyterlab jupyter-lsp# JupyterLab 2.x
jupyter labextension install @krassowski/jupyterlab-lsp# JupyterLab 1.x
jupyter labextension install @krassowski/jupyterlab-lsp@0.8.0
Acknowledgements
The development of the PYLS-Memestra was possible thanks to the amazing work of Serge “sans Paille” Guelton on Memestra, Beniget and Gast and Michal Krassowski and Nicholas Bollweg on the JupyterLab-LSP as well as the whole open source community around the projects.
About the author
My name is Mariana Meireles and I’m a software developer working for QuantStack. I care deeply about the impacts that technology has in the world and try my best to be the change I want to see by contributing to open source projects that stand upon libre and diverse standards.
Prior to QuantStack I worked as a developer on the PySide team at the Qt Company and as a web performance developer at Mozilla.
