You have probably wondered at some point why most developers choose Unix based operating systems such as Linux distributions or macOS over Windows. The root cause of the issue can be simply reduced to the availability of the development technologies, meaning that Windows does not and cannot offer some features that are vital to some development processes. If this is the fact, does Windows acknowledge this issue, and what do they do about this? At Picnic, as an OS-agnostic company, how do we resolve the problems arising from using multiple operating systems for development?

The differences in operating systems from both architectural and technical points of view result in countless problems in software engineering. In Picnic, we have managed to standardize the Python development environment for both Unix based Linux/macOS and Windows users. This unified working experience would not be possible if it were not for WSL, Windows Subsystem for Linux. In this article, I will dive into the Windows development environment in which various Unix based tools are installed.

Unix and Windows

You probably read or heard that someone considers a problem “Windows related”. It can be in the changelog of a tool, in a bug report, or as simple as someone reacting to a problem you have in your Windows machine. What does that mean? Some games are only released for Windows. Some software technologies are only available in Unix based operating systems. These situations occur because of various reasons. The underlying technologies these OS’es are built upon are some of them. One can think of them as “magical components” that make up an operating system such as the file systems, kernels, drivers, pre-built compilers, binaries, etc. Another reason is the target audience for which the organizations build their operating systems. An average Windows user can be considered as a consumer that seeks an end-user ready product. Windows hides away the underlying technology and polish their UI for this reason. On the other hand, the most popular UNIX based operating systems like Ubuntu and Arch Linux are more direct, give more freedom, and expose underlying technology.

In Picnic, we have a broad project called Edge Systems where the business analysts, who strive for the initiative, have the opportunity to develop cronjob-like services and use tools to enhance and deploy them automatically. This enables them to reach their maximum potential as analysts by increasing their technical capacity and automating repetitive tasks. There are enabler tools that we develop internally varying from templates, code linters to CICD pipelines. There are also some tools over which we don’t have full control such as pyenv, pipenv and Python itself. All these tools are developed in Unix environments. The core problem of not being able to support everyone arises from this point because Picnic is an OS-agnostic workplace. Most analysts use Windows whereas most software engineers prefer Unix based operating systems. Due to the nature of the difference in operating systems, the tools cannot behave the same in different operating systems. This always results in problems, support tickets, questions about tools acting differently in multiple operating systems.

This is where WSL comes into play.

The solution: WSL

Windows Subsystem for Linux (WSL) is a technology being developed by Microsoft that lets you create a Linux VM that is perfected to work within Windows 10. While doing so, Microsoft admits the problem they have and addresses the issues mentioned above by developing WSL, essentially enabling users to use Unix based technologies. The following is from the official WSL FAQ:

This is primarily a tool for developers — especially web developers and those who work on or with open source projects. This allows those who want/need to use Bash, common Linux tools (sed, awk, etc.) and many Linux-first tools (Ruby, Python, etc.) to use their toolchain on Windows.

It is worth denoting that some controversies imply the development of WSL is a way for Microsoft to embrace, extend, and extinguish Linux.

One can think of WSL as a daemon process always running in the background. It offers not a GUI but command-line access in which Linux commands can be run. Should someone want to use some technology only available in Linux, they can install it on WSL via this command-line interface. Moreover, it has a direct connection to the Windows 10 file system and vice versa. This direct and fast connection between Windows and WSL allows developers to build programs on the Windows side that execute commands on the WSL side and get its output.

There are two WSL major releases, WSL 1 and WSL 2. The latter offers more features and optimizations whereas the former has the edge in being more performant across file systems. One big advantage of WSL 2 that piques most developers’ interest is its built-in support for Docker. This means that if you have Docker Desktop installed on Windows, it can be available on the WSL side as well without any further effort. Read more about how Docker developers explain their WSL integration here.

The lack of GUI can be interpreted as a prime problem as it makes it harder to write code. The only option seems to use command-line tools such as vi/m , nano , etc. However, it is not the case. Like mentioned before, Unix commands can be called in WSL from the Windows side. This enables IDEs to support WSL in a way that you can continue writing code via your IDE on the Windows side and run it on the WSL side. Furthermore, the Windows team is working on GUI support for applications residing in WSL.

Now that the importance of WSL and how effective it can be in a developer’s arsenal are clear, we can explore two widely used IDEs and their support in WSL, namely PyCharm and VSCode. Let us dive into this research topic that is carried out in the scope of migration to WSL.

Pycharm

Pycharm is one of the most popular IDEs. With its features specifically tailored for Python development which come out of the box without any extra installations, it provides every need of a Python developer to them. Due to these builtin features, it can be slower than its alternatives.

The latest version of Pycharm (2020.2 as of the writing) supports WSL integration only to some extent. There are many open issues and undergoing developments. The integration does not seem to reach its maturity and it is easy to find bugs while trying it out.

• Pycharm allows you to select a Python interpreter that is in WSL. The interpreter is immediately used for file indexing which is relatively slow and can sometimes take approximately 10 minutes.
• Pycharm requires the project files to be on the Windows side to view and edit them. For this reason, git needs to be installed on the Windows side.
• To run or debug a piece of code, Pycharm generates a shell command to tell WSL to run a Python command on the files in Windows. That is possible as WSL has access to the Windows file system. However, we discovered some bugs in some of these commands such as wrong character escaping and misplacement of environment variables in the command.
• The Docker GUI in Pycharm uses Docker from Windows.
• At Picnic, we use some tools that expect the project files to stay in the same environment as the Python interpreter resides in. Therefore, the code repository is also cloned on the WSL side which results in a particularly hard problem to overcome. The code is replicated and the changes on both sides should be git-tracked and synchronized constantly.

Moreover, Pycharm offers this integration only in their professional edition which is not free. Nonetheless, if you do not encounter bugs or work around them, it works.

To sum up, since Pycharm does not support remote development and due to its communication design with WSL, you can only do so much. Even though the development is still doable, it is definitely worth looking into its alternatives.

VSCode

With the increasing number of developers that adapt to work with VSCode, the IDE gains more recognition in the Python community each day. VSCode is a lightweight code editor that is aimed at no specific programming language. Extensions can be installed to equip the editor with more features.

VSCode has an interesting and efficient integration design for WSL. Probably thanks to the fact that both VSCode and WSL are being developed by Microsoft, it is no wonder that they go hand in hand.

• VSCode has an extension (“Remote — WSL”) to enable WSL support. Once you run code on the WSL side, it installs a server so that the VSCode on the Windows side can communicate with its WSL counterpart. After the server is installed, the VSCode on the Windows side is only for UI and code editing purposes. It can be considered as a window into the VSCode on the WSL side. For instance, when installing further extensions such as Python or Docker, they are installed not on the Windows side but on the WSL side.
• VSCode, in contrast to Pycharm, can use the file system on WSL, open a project based in WSL and edit them. code . command can be run in the WSL terminal to open a project in a new VSCode tab on the Windows side.

• Development is quite fast including indexing speed as the actual VSCode server resides on the WSL side.
• VSCode offers a series of quality-of-life improvements that automate some processes such as automatic Python interpreter discovery and fast debugger.

In conclusion, VSCode seems to master the integration with WSL thanks to its superior architecture. It would not be wrong to say the integration will bear more fruits in future as WSL improves. VSCode makes the developer feel a seamless Unix development experience in Windows! For all these reasons, we opted for VSCode in our Windows development environments.

The ability to use Unix tools smoothly in Windows is the perfect solution. Once we discovered what WSL has to offer and its integration with popular IDEs, it was inevitable to continue researching further. And the research demonstrated that we can set up all the tools required in Windows!

Python Development Environment

So far, we have talked about the problems, solutions, and trade-offs to make some tools and technologies to work in Windows to be able to develop with Python. So, what kind of tools and technologies are we using?

• Python: We always use the latest versions of Python!
• pyenv: This is how we manage different Python distributions.
• pipenv: Virtual environment, dependency management, and dependency locking.
• git: The one and only source control.
• Docker: The one and only containerization tool.
• Picnic Tools: Custom tools that we internally develop that serve various purposes such as automation, linting, private packages, etc.

Some Learnings

I was prejudiced against using an IDE other than Pycharm, namely VSCode. However, when I researched VSCode to determine its WSL capabilities I got surprised with its overall abilities. Consequently, I switched from Pycharm to VSCode and have been using it since then. Be open to change and do not judge without trying.

Picnic is a scale-up where engineers are constantly working to simply make things better. They are given opportunities to research cutting-edge technologies and come up with solution proposals to existing problems even though it may lead to company-wide adoption. We have found out that WSL along with the other tools mentioned solve a lot of our problems. Do not hesitate to research and evaluate new technologies.

Conclusion

The added value of WSL is not measurable since it offers something new that we use to achieve the standardization of tools and unification of programming in various operating systems. Moreover, development tools like Docker, Pycharm, VSCode are constantly working to integrate their systems with it. It may transform how people think about developing in Windows. Although in Picnic we use it to set up a Python development environment, its reach can easily be extended by simply installing other tools just like one could install in Linux.