A Brief History of Unix Commands On Windows: CoreUtils (Again)

If you use Windows today and type ls , cat , grep , or awk in a terminal, there is a good chance something useful will happen. That was not always true. For most of the history of personal computing, Unix/Linux and Windows lived on opposite sides of a cultural border. Unix people had pipes, small composable tools, shell scripts, make , sed , awk , grep , tar , and the idea that everything was a file. Windows people had drive letters, backslashes, COMMAND.COM or cmd.exe, and an API that did not care much about what POSIX thought. Yet there has always been a demand for Unix tools on Windows. Some of it came from programmers who wanted the same build scripts everywhere. Some came from administrators who missed grep and awk . Some came from companies trying to port big Unix applications to NT without rewriting them all. The result is a long, strange history of Unix-on-Windows layers, toolkits, compromises, and almost-but-not-quite compatibility. Easy? The simplest version of the problem sounds trivial. How hard can cat be? Open a file, copy bytes to standard output, done. Writing ls is a little more work, but Windows has directory APIs. Common commands like cp , mv , rm , mkdir are not very mysterious. Even pipes are not foreign to Windows. A lot of the everyday Unix command set can be ported as ordinary Win32 console programs with some path handling and enough patience. But not all of Unix or Linux translates cleanly to Windows. The big issue is fork() . On Unix, a process can clone itself. The child gets a copy of the parent’s address space, open file descriptors, environment, signal state, and so on. Modern kernels make this efficient with copy-on-write memory, but the programming model is old and deeply baked into Unix. Shells use it constantly. Servers use it. Build systems use it. Scripting languages assume it exists, or at least that the surrounding environment behaves as though it does. Windows process creation is different. Windows has CreateProcess() , which starts a new program. That is a perfectly reasonable model, but it is not fork() (more like fork()+exec() ). If you are just launching notepad.exe , no problem. If you are trying to implement a POSIX shell that forks, redirects file descriptors, adjusts the environment, and then starts another program, the mismatch is extreme and you’ll have to do some strange things to fake things out. One of the early commercial answers was the MKS Toolkit , originally from Mortice Kern Systems. MKS gave Windows users a pile of familiar commands, shells, and development tools. It was not just ls and friends; it included things like ksh , vi , grep , find , awk , make , and many of the utilities needed to move scripts and build procedures between Unix and Windows. The current PTC MKS documentation still describes it in exactly that spirit: Unix shells and hundreds of commands for interoperability with Windows. MKS was attractive because it treated Windows as Windows. You were not necessarily pretending your machine was a Unix workstation. You were getting a Unix-flavored toolbox that could operate in a Windows environment. For many people, that was enough. You could write scripts, process text, drive builds, and avoid learning three different syntaxes for the same job. AT&T Returns A more ambitious attempt came from AT&T Bell Labs: UWIN , a pet project of David Korn of KornShell fame. UWIN stood for Unix for Windows, and that is more or less what it tried to be: a Unix interface layer on top of Windows NT and Windows 95, with libraries, headers, and commands. Korn’s 1997 USENIX paper describes the goal as building an open environment rich enough to serve as both a development and an execution environment, with nearly all of the X/Open Release 4 headers, interfaces, and commands. UWIN is one of those projects that feels both obvious and heroic. Obvious, because if you have a bunch of Unix software, you want a compatibility layer. Heroic, because every POSIX semantic eventually has to land on some Windows behavior that was designed with different assumptions. File permissions, symbolic links, signals, terminals, process groups, device files, pathnames, and fork() all have edge cases. You can make the common case work and still spend years fighting the uncommon cases. Cygwin Then there is Cygwin, probably the most famous of the classic solutions. Cygwin became the go-to answer for people who wanted GNU tools and a fairly complete POSIX environment on Windows. The model is different from simply compiling ls.exe as a native Windows program. Cygwin provides a POSIX compatibility DLL. Programs built for Cygwin call into that layer, and the layer translates Unix expectations into Windows operations. That is both Cygwin’s strength and its weakness. It is strong because a lot of Unix software can be rebuilt and made to run. The Cygwin project is explicit that it is not a way