From syntax highlighting to code completion, a complete Python IDE inside Jupyter
This article is part of a series. Check out the full series: Part I, Part II, Part III.
Over the past few days, we’ve been building a complete Python IDE inside Jupyter. In this article, we will add the final touches and package everything in a Docker image to create a portable working environment for data scientists and Machine Learning engineers.
Jupyter is not exactly an IDE. It’s not even an IPython UI, as many may think. I would argue that Jupyter is a development platform, and you can create your own workspace the way you like it. Since it includes a terminal emulator, you can do anything you may think. Thus, we will use it to create a complete, feature-packed Python IDE.
Thus, in previous articles, we installed Neovim and configured it to act and look similar to the most popular text editor these days: Visual Studio Code. In this article, we will install one tool to conquer all: code completion, code formatting, git integration, spell checking, and lining. Let’s begin!
Learning Rate is a newsletter for those who are curious about the world of ML and MLOps. If you want to learn more about topics like this subscribe here. You’ll hear from me on the last Sunday of every month with updates and thoughts on the latest MLOps news and articles!
Many tools promise to make a developer’s life easier when it comes to Vim/Neovim, but my preference is Conquer of Completion.
Conquer of Completion (CoC) is a popular plugin for Vim and Neovim that provides a powerful autocompletion engine. Here are a few key things to know about CoC:
- CoC supports multiple languages: it can provide autocompletion suggestions for various programming languages, including Python, JavaScript, C, Go, and many others. It uses language servers to provide intelligent recommendations based on your code context.
- CoC is highly configurable: it allows users to customize the autocompletion engine to fit their preferences in a VS Code-like way. Users can set up custom mappings for triggering completion, adjust the priority of completion sources, and more.
- CoC has a large user community: it is a popular plugin with a large user community, which means many resources are available for troubleshooting and customization. The plugin is also actively maintained and updated. Moreover, the community provides several great extensions we will use.
Overall, Conquer of Completion is a powerful plugin that can significantly enhance your autocompletion experience in Vim or Neovim, mainly if you work with multiple programming languages.
CoC requires some setup. You’ll need to install and configure some additional components, including NodeJS and the npm package manager. Depending on your case, you may also need to install a language server for your preferred programming language.
In our case, let’s first install NodeJS.
NodeJS Installation
NodeJS and npm are hard requirements to install the CoC plugin for Neovim and enable auto-completion for many programming languages.
To install the two packages execute the following commands:
- Install the PPA to get access to its packages:
curl -sL https://deb.nodesource.com/setup_18.x -o nodesource_setup.sh
sudo bash nodesource_setup.sh
2. Install Node.js:
sudo apt install nodejs
3. Verify:
node -v
If you get something like v18.15.0 (the latest version at the time of writing this article) you’re good to go. But even if you have another version installed, you should be able to continue.
Install CoC
Installing the CoC plugin is as simple as any other plugin. Just place the following line in your init.vim configuration file:
Plug 'neoclide/coc.nvim', {'branch': 'release'} " Auto complete Python
Restart the editor and run the :PlugInstall Vim command. vim-plug, the plugin manager we are using will install CoC for you. To complete the installation process, restart the editor again.
To see the installation process of a plugin in Neovim in detail, refer back to the second blog of this series.
We are ready to install the extensions we need to turn our editor into a full-fledged IDE. Run the :CocInstall coc-pyright Vim command to install support for Python. After finishing the installation, quit every window of your editor and open a Python file like this:
nvim test.py
Autocomplete should be enabled; however, it’s not very practical. So, let’s configure CoC. Add the following lines in your init.vim configuration file:
" CoC customization" Applying code actions to the selected code block
vmap <leader>a <Plug>(coc-codeaction-selected)<cr>
xmap <leader>a <Plug>(coc-codeaction-selected)<cr>
" Format action on <leader>
vmap <leader>f <Plug>(coc-format-selected)
xmap <leader>f <Plug>(coc-format-selected)
" GoTo code navigation
nmap <silent> gd <Plug>(coc-definition)
nmap <silent> gv :vsp<CR><Plug>(coc-definition)<C-W>L
nmap <silent> gy <Plug>(coc-type-definition)
nmap <silent> gi <Plug>(coc-implementation)
nmap <silent> gr <Plug>(coc-references)
" Symbol renaming
nmap <leader>rn <Plug>(coc-rename)
" Refactor
nmap <silent> <leader>re <Plug>(coc-codeaction-refactor)
xmap <silent> <leader>r <Plug>(coc-codeaction-refactor-selected)
nmap <silent> <leader>r <Plug>(coc-codeaction-refactor-selected)
" Confirm selection by pressing Tab
inoremap <expr> <Tab> pumvisible() ? coc#_select_confirm() : "<Tab>"
" Open autocomplete menu by pressing Ctrl + Space
inoremap <silent><expr> <c-space> coc#refresh()
" Open documentation by pressing K
nnoremap <silent> K :call ShowDocumentation()<cr>
function! ShowDocumentation()
if CocAction('hasProvider', 'hover')
call CocActionAsync('doHover')
else
call feedkeys('K', 'in')
endif
endfunction
I left a comment for each line so you can understand what it does. Now CoC is configured, and you can use these key bindings to get the most out of it.
However, things do not end here. As we said, CoC has several extensions (like coc-pyright) that you can install. Some of them are:
coc-gitcoc-dockercoc-yamlcoc-jsoncoc-prettiercoc-pairscoc-spell-checker
To install any of them, just run the :CocInstall command and provide the name of the extension. For a complete list of extensions, visit the coc-extensions page: https://github.com/topics/coc-extensions
Finally, you can configure CoC much like you do with VS Code. Run the :CocConfig command and the coc-settings.json file will pop up. I have the following settings set there:
{
"python.venvPath": "/home/dimpo/.pyenv/versions/",
"python.linting.flake8Enabled": true,
"python.formatting.provider": "autopep8",
"pyright.inlayHints.variableTypes": false,
"pyright.inlayHints.functionReturnTypes": false,
"git.addGBlameToVirtualText": true,
}
Each row is self-explanatory, but if you want to find out what exactly you can configure, refer to the documentation of each CoC extension.
As usual, save your work using the docker commit command (refer to previous articles in this series if you don’t know what this means). You can find my image in DockerHub.
In this series of articles, we built a complete Python IDE inside a JupyterLab environment. We started with a simple Neovim installation, which we configured to act and look like the most popular text editor today, VS Code.
This article concludes this story, but if you want to see more things, like how to use a debugger or configure a specific plugin further, please leave a comment, and I’ll do my best! Till then, happy coding.
My name is Dimitris Poulopoulos, and I’m a machine learning engineer working for Arrikto. I have designed and implemented AI and software solutions for major clients such as the European Commission, Eurostat, IMF, the European Central Bank, OECD, and IKEA.
If you are interested in reading more posts about Machine Learning, Deep Learning, Data Science, and DataOps, follow me on Medium, LinkedIn, or @james2pl on Twitter.
Opinions expressed are solely my own and do not express the views or opinions of my employer.
From syntax highlighting to code completion, a complete Python IDE inside Jupyter
This article is part of a series. Check out the full series: Part I, Part II, Part III.
Over the past few days, we’ve been building a complete Python IDE inside Jupyter. In this article, we will add the final touches and package everything in a Docker image to create a portable working environment for data scientists and Machine Learning engineers.
Jupyter is not exactly an IDE. It’s not even an IPython UI, as many may think. I would argue that Jupyter is a development platform, and you can create your own workspace the way you like it. Since it includes a terminal emulator, you can do anything you may think. Thus, we will use it to create a complete, feature-packed Python IDE.
Thus, in previous articles, we installed Neovim and configured it to act and look similar to the most popular text editor these days: Visual Studio Code. In this article, we will install one tool to conquer all: code completion, code formatting, git integration, spell checking, and lining. Let’s begin!
Learning Rate is a newsletter for those who are curious about the world of ML and MLOps. If you want to learn more about topics like this subscribe here. You’ll hear from me on the last Sunday of every month with updates and thoughts on the latest MLOps news and articles!
Many tools promise to make a developer’s life easier when it comes to Vim/Neovim, but my preference is Conquer of Completion.
Conquer of Completion (CoC) is a popular plugin for Vim and Neovim that provides a powerful autocompletion engine. Here are a few key things to know about CoC:
- CoC supports multiple languages: it can provide autocompletion suggestions for various programming languages, including Python, JavaScript, C, Go, and many others. It uses language servers to provide intelligent recommendations based on your code context.
- CoC is highly configurable: it allows users to customize the autocompletion engine to fit their preferences in a VS Code-like way. Users can set up custom mappings for triggering completion, adjust the priority of completion sources, and more.
- CoC has a large user community: it is a popular plugin with a large user community, which means many resources are available for troubleshooting and customization. The plugin is also actively maintained and updated. Moreover, the community provides several great extensions we will use.
Overall, Conquer of Completion is a powerful plugin that can significantly enhance your autocompletion experience in Vim or Neovim, mainly if you work with multiple programming languages.
CoC requires some setup. You’ll need to install and configure some additional components, including NodeJS and the npm package manager. Depending on your case, you may also need to install a language server for your preferred programming language.
In our case, let’s first install NodeJS.
NodeJS Installation
NodeJS and npm are hard requirements to install the CoC plugin for Neovim and enable auto-completion for many programming languages.
To install the two packages execute the following commands:
- Install the PPA to get access to its packages:
curl -sL https://deb.nodesource.com/setup_18.x -o nodesource_setup.sh
sudo bash nodesource_setup.sh
2. Install Node.js:
sudo apt install nodejs
3. Verify:
node -v
If you get something like v18.15.0 (the latest version at the time of writing this article) you’re good to go. But even if you have another version installed, you should be able to continue.
Install CoC
Installing the CoC plugin is as simple as any other plugin. Just place the following line in your init.vim configuration file:
Plug 'neoclide/coc.nvim', {'branch': 'release'} " Auto complete Python
Restart the editor and run the :PlugInstall Vim command. vim-plug, the plugin manager we are using will install CoC for you. To complete the installation process, restart the editor again.
To see the installation process of a plugin in Neovim in detail, refer back to the second blog of this series.
We are ready to install the extensions we need to turn our editor into a full-fledged IDE. Run the :CocInstall coc-pyright Vim command to install support for Python. After finishing the installation, quit every window of your editor and open a Python file like this:
nvim test.py
Autocomplete should be enabled; however, it’s not very practical. So, let’s configure CoC. Add the following lines in your init.vim configuration file:
" CoC customization" Applying code actions to the selected code block
vmap <leader>a <Plug>(coc-codeaction-selected)<cr>
xmap <leader>a <Plug>(coc-codeaction-selected)<cr>
" Format action on <leader>
vmap <leader>f <Plug>(coc-format-selected)
xmap <leader>f <Plug>(coc-format-selected)
" GoTo code navigation
nmap <silent> gd <Plug>(coc-definition)
nmap <silent> gv :vsp<CR><Plug>(coc-definition)<C-W>L
nmap <silent> gy <Plug>(coc-type-definition)
nmap <silent> gi <Plug>(coc-implementation)
nmap <silent> gr <Plug>(coc-references)
" Symbol renaming
nmap <leader>rn <Plug>(coc-rename)
" Refactor
nmap <silent> <leader>re <Plug>(coc-codeaction-refactor)
xmap <silent> <leader>r <Plug>(coc-codeaction-refactor-selected)
nmap <silent> <leader>r <Plug>(coc-codeaction-refactor-selected)
" Confirm selection by pressing Tab
inoremap <expr> <Tab> pumvisible() ? coc#_select_confirm() : "<Tab>"
" Open autocomplete menu by pressing Ctrl + Space
inoremap <silent><expr> <c-space> coc#refresh()
" Open documentation by pressing K
nnoremap <silent> K :call ShowDocumentation()<cr>
function! ShowDocumentation()
if CocAction('hasProvider', 'hover')
call CocActionAsync('doHover')
else
call feedkeys('K', 'in')
endif
endfunction
I left a comment for each line so you can understand what it does. Now CoC is configured, and you can use these key bindings to get the most out of it.
However, things do not end here. As we said, CoC has several extensions (like coc-pyright) that you can install. Some of them are:
coc-gitcoc-dockercoc-yamlcoc-jsoncoc-prettiercoc-pairscoc-spell-checker
To install any of them, just run the :CocInstall command and provide the name of the extension. For a complete list of extensions, visit the coc-extensions page: https://github.com/topics/coc-extensions
Finally, you can configure CoC much like you do with VS Code. Run the :CocConfig command and the coc-settings.json file will pop up. I have the following settings set there:
{
"python.venvPath": "/home/dimpo/.pyenv/versions/",
"python.linting.flake8Enabled": true,
"python.formatting.provider": "autopep8",
"pyright.inlayHints.variableTypes": false,
"pyright.inlayHints.functionReturnTypes": false,
"git.addGBlameToVirtualText": true,
}
Each row is self-explanatory, but if you want to find out what exactly you can configure, refer to the documentation of each CoC extension.
As usual, save your work using the docker commit command (refer to previous articles in this series if you don’t know what this means). You can find my image in DockerHub.
In this series of articles, we built a complete Python IDE inside a JupyterLab environment. We started with a simple Neovim installation, which we configured to act and look like the most popular text editor today, VS Code.
This article concludes this story, but if you want to see more things, like how to use a debugger or configure a specific plugin further, please leave a comment, and I’ll do my best! Till then, happy coding.
My name is Dimitris Poulopoulos, and I’m a machine learning engineer working for Arrikto. I have designed and implemented AI and software solutions for major clients such as the European Commission, Eurostat, IMF, the European Central Bank, OECD, and IKEA.
If you are interested in reading more posts about Machine Learning, Deep Learning, Data Science, and DataOps, follow me on Medium, LinkedIn, or @james2pl on Twitter.
Opinions expressed are solely my own and do not express the views or opinions of my employer.