Library Infrastructure Project

- - 2003-09-12 - Library Infrastructure Project - - Isaac - Jones - - - Simon - Peyton Jones - - - Simon - Marlow - - - Malcolm - Wallace - - -

ijones@syntaxpolice.org

- - - The Library Infrastructure Project is an effort to provide - a framework for developers to more effectively contribute their - software to the Haskell community. - - The &impls;Herein, I will use &impl; as a - catch-all phrase to include compilers and interpreters for the - Haskell programming language. come included - with a good set of standard libraries, but this set is - constantly growing and is maintained centrally. This model does - not scale up well, and as Haskell grows in acceptance, the - quality and quantity of available libraries is becoming a major - issue. There are a wide variety of &impls; (both compilers and - interpreters), each of which target a variety of hardware - architectures and operating systems. There are also a number of - different groups and individuals providing libraries, and these - libraries depend upon each other and they depend upon external - systems such as Java or GTK. - - It can be very difficult for an end user to manage these - dependencies and build all the necessary software at the correct - version numbers on their platform: there is currently no generic - build system to abstract away differences between Haskell - Implementations and operating systems Your - humble author has in-depth knowledge of Debian GNU/Linux, but - could probably learn a thing or two about other operating - systems like Windows and MacOS. Should you notice anything here - which offends your sense of operating system, please let me - know.. - - The Library Infrastructure Project seeks to provide some - relief to this situation by building tools to assist developers, - end users, and operating system distributers. - - This is a draft proposal. If you have comments, please - email Isaac - Jones. The latest version of this document should be - available in a variety of formats from the Library - Infrastructure Project home - page. This proposal is seperate, but related. - - - - - - - -The Haskell Library Infrastructure: goals - -The Haskell Library Infrastructure (HLI) has the following main goal: -to specify a standard way in which a Haskell library can be packaged, so that it is -easy for consumers to use it, or re-package it, -regardless of the Haskell implementation or installation platform. - - We need to come up w/ an agreed-upon name. For one thing, the - entire "infrastructure" will not only include the above, but also a - web site for uploading packages, etc. - -The HLI also -supports library authors by providing an instrastructure that automates the -process of building and packaging simple libraries. It is not necessary to use -this code --- indeed complex libraries may exceed its abilities --- but it should -handle 75% of the cases with no trouble. - -The HLI serves a number of different people: - - - Joe User is simply a Haskell user. He does not download new pacakges. Nevertheless, -he needs to know about his Haskell compiler's -package flag (see ). - - - - Bob the Builder and Sam Sysadmin both download, build, - and install new pacakges. The only difference between the two is that Sam has root permission, - and can install pacakges in more globally-visible places. - - - - Roland RPM, Donald Debian, and Willie Windows - build Linux RPM, Debian, and Windows installer packages respectively. (This list is not exhaustive.) - They do this as a service for Angela Author and the community, and may know little or nothing about the internal details - of the Haskell packages they are wrapping up. - - - - Angela Author wants to write a simple Haskell library, and distribute it with -minimum fuss, in such a way that all the above folk can easily use it. - - - - Marcus Makefile is like Angela, but more sophisticated. He has a complicated library, -and uses makefiles. Still, he wants to arrange that Roland, Donald, Bob, Sam, and Joe don't need to know -about his internal complexity. - - - -We describe Angela and Marcus as producers of their packages, and all the others -as package consumers. - - - - -The Haskell Library Infrastructure: overview - -This section summarises the vocabulary and main features of the Haskell Library Infrastructure. - -Packages - -A package is the unit of distribution for HLI. -Each package has a package description (PD) -that consists of: - -A globally-unique package name, containing no spaces. Chaos will result -if two distinct packages with the same name are installed on the same system. We need a -global web server where authors can claim and record package names. - -A version, consisting of a sequence of one or more integers. - - - -A list of explicit dependencies on other packages. These are typically not exact; e.g. "I need hunit version -greater than 2.4". - - - -A list of exposed modules. Not all of the modules that comprise a package implementation are necessarily -exposed to a pacakge client. -The ability to expose some, but not all, of the modules making up a package -is rather like using an explicit export list on a Haskell module. - >This isn't specified in - - -More detailed information about the make-up of the package and how to build it. -The full details are given in . - - -The first two components can be combined to form a single text string called the package ID, -using a hyphen to separate the version -from the name, and dots to separate the version components. For example, "hunit-2.3". - - - - -Package distributions - -A HLI package can be distributed in several forms: - - -A HLI source distribution is a tree of files (tar-ball, zip file etc) -containing the library sources, which may need to be -compiled before being installed. The same source tarball may well be installable for several -Haskell implementations, OSs, and platforms. - -A source distribution may contain fewer files than appear in the developer's CVS repository; for example, -design notes may be omitted. It may also contain some derived files, that do not appear in the -the developer's repository; for example, ones made by a somewhat exotic pre-processor where it -seems simpler to ship the derived file than to ensure that all consumers have the pre-processor. - - - -A HLI binary distribution is a tree of files that contains a pre-compiled library, ready -for installation. The pre-compilation means that the distribution will be Haskell-compiler-specific, and certain "looser" dependencies (hunit > 2.3) will now be precisely fixed (hunit == 2.4). - - - - -The library may be packaged as an RPM, Debian distribution, -or Windows installer. (This list is not exhaustive.) -In that case, the way it is installed is prescribed by the respective distribution mechanism; -the only role of the HLI is to make it easy to construct such distributions. All three are -compiler-specific binary distributions. - - - - - -Compilers - -We use the term ``compiler'' to mean GHC, Hugs, nhc, hbc, etc. (Even though -Hugs isn't really a compiler, the term is less clumsy than ``Haskell implementation''.) - - The -HLI requires that a conforming Haskell compiler is somewhat package aware. -In summary, the requirements are these: - -Each compiler hc must provide an associated package-management -program hc-pkg. A compiler user installs a package by placing the package's -supporting files somewhere, and then using hc-pkg to make the compiler aware -of the new package. - -Subsequent invocations of hc will include modules from the -new package in the module name space (i.e. visible to import statements). - - -The compiler should support a -package flag for finer-grain control -of package visibility. - - -A complete specification of these requirements is given in . - - - -The Setup script - -The key question is this: how should Angela Author present her HLI package so that her consumers (from -Bob, Sam, Willie, etc) can conveniently use it? - -Answer: she provides a tree of files, with the file Setup.lhs in the -root directory of the tree. Setup.lhs is an executable Haskell program -which conforms to a particular specification, given in detail in . -In summary, though, Setup.lhs allows a consumer to configure, build, -install, register, and un-register a package. - For example, consider a package hunit-2.3. -Bob the Builder downloads and unpacks a source tar-ball into a temporary directory, -cd's to root directory, and - - ./Setup.lhs configure --ghc - ./Setup.lhs build - ./Setup.lhs install - - -Should we state the rule about when one must explictly call "register"? - -The configuration step checks, for example, that the package author indeed intends the package to be buildable -with GHC, that a sufficiently up-to-date GHC is available, that the packages which are required to build hunit-2.3 are already installed for GHC, and so on. -After that the build and install steps know what -compiler they are using. - -Question: can you abbreviate to Setup install ghc? - -In principle, the Setup program -could be written in any language; see for why we use Haskell. - - - - -Help for building packages - -The HLI allows a package author to write the setup script in any way she pleases, provided -it conforms to the speciication of . However, many Haskell packages consist -of little more than a bunch of Haskell modules, and for these the HLI provides a library that does -all the work. -For example, suppose Angela Author has a bunch of Haskell modules Angela.Set, -Angela.Bag, and Angela.Internals, that she wants to -turn into a package. All she need do is write the setup script - - #! /usr/bin/env runhugs - import Distribution.Simple - - pkgDescr = emptyPackage { name = "angela", - version = [1], - modules = ["Angela.Set", "Angela.Bag"] } - - main = defaultMain pkgDescr - -Now she can use ./Setup.lhs sdist to create a source distribution. - -The details of Distribution.Simple are given in . - - - - - - But Why Should We Use Haskell? - -The setup script, Setup.lhs, is written in Haskell. Why? - - -Haskell runs on all the systems of interest. - -Haskell's standard libraries should include a rich set of operating -system operations needed for the task. These can abstract-away the -differences between systems in a way that is not possible for -Make-based tools. - -Haskell is a great language for many things, including tasks -typically relegated to languages like Python. Building, installing, -and managing packages is a perfect proving ground for these tasks, and -can help us to discover weaknesses in Haskell or its libraries that -prevent it from breaking into this "market". A positive side-effect -of this project might be to make Haskell more suitable for "scripting" -tasks. - -Likewise, each piece of the project (Building, Installing, and -Packaging) can be leveraged elsewhere if we make them into -libraries. - -Make is not particularly good for parsing, processing, and sharing -meta-information about packages. The availability of this information -to Haskell systems (including compilers, interpreters, and other -tools) is useful. Unlike Make, Haskell can also reuse unrelated -algorithms, parsers, and other libraries that have been developed in -the past. - -Dogfooding, the act of using the tools you -develop, is a healthy policy. - - -It is convenient for consumers to execute Setup.lhs directly, thus: - - ./Setup.lhs ... - -This can be achieved by starting Setup.lhs with "#! /usr/bin/env runhugs". -Since it's a literate Haskell script (.lhs file), the Haskell compiler will ignore -this line. -However, nothing stops a consumer from running the script interactively, or compiling it and running -the compiled binary. Any implementation of Haskell should suffice to run the script, provided -the implementation has the HLI libraries installed. - - The GHC team should provide runghc. - - - - - -Package consumers and the setup script - - -The sole requirement of an HLI package is that it should contain a setup script, Setup.lhs, -in the root of its file structure. This section -specifies the complete command-line interface for the setup script, and how it satisfies the needs of -package consumers. - -The setup script specification - -Here is the command-line interface the setup script must satisfy. - setup.lhs interface - - - - ./Setup.lhs info - Output the package description. - - - - ./Setup.lhs configure [flags] - Prepare to build the package. Typically, this step checks - that the target platform is capable of building the package, and - discovers platform-specific features that are needed during the build. - - - - ./Setup.lhs build - Make this package ready for installation. For a true compiler, - this step involves compiling the Haskell source code. Even for an interpreter, however, - it may involve running a pre-processor. - - - ./Setup.lhs install [install-prefix] - Copy the files into the install locations, and register - the package with the compiler. - - - ./Setup.lhs register - ./Setup.lhs unregister - - - Register (or un-register) this package with the - compiler. (NB: registration is also done automatically by install.) - - - ./Setup.lhs test - Run the package's test suite. - - -

- - -<function>setup info</function> - -The command ./Setup.lhs info outputs the package description on the -standard output, in the format given below. This the only concrete way in which the package -descrption appears in the HLI: Angela Author does not write a file containing the package description. - -Insert package desription syntax here. - -NB: the package descrpition is not the same as an installed package description (); -the latter describes a particular -installation (e.g. where the files are), while PkgDescr gives only installation-independent information. - - - - -<function>setup configure</function> - -The command ./Setup.lhs configure prepares -to build the package. For sophisticated packages, the configure step -may perform elaborate checks, to gather information about the target -system. It may write a file to record its results, but the name and -format of the file are not part of the specification. For wrapped -make-based systems (for instance), a command-line parser that -understands the standard targets will be provided. - - - -All flags are optional. The flags are these: - ---with-compiler=path, ---ghc, ---nhc, ---hugs: -specifies which compiler to use. At most one of the value of these flags may be specified. -The configure step checks -that the compiler is available, in a sufficiently up-to-date form for the package, and that the package -expects to work with that compiler. If the compiler name -is not specified, setup will choose one; some packages will come with one compiler baked in. - - ---prefix=path: specifies where the installed files -for the package should be installed. Typically on Unix this will be /usr/local and -on Windows it will be Program Files. The setup script will use a sensible default -(often platform-specific) if the flag is not specified. - - - -Unrecognized flags are errors in the default build system, but may be meaningful to wrapped make-based systems (for instance). Therefore, the provided command-line parser will pass unrecognized command-line flags on to the wrapped system. - - -It is OK for these flags to be "baked into" the compiled library. In particular, the build system may -bake the installation path into the compiled files. There is no provision for installing the compiled -files anywhere other than the place specified in the configure step. - - - -<function>setup build</function> - -The command ./Setup.lhs build makes this -package, ready for installation. It takes no flags. - - - -<function>setup install</function> - -The command ./Setup.lhs install copies files from the built package to -the right location for installed files, specified in the configure step. Then it registers the new package with -the compiler, using the hc-pkg command. - ---install-prefix=path has three effects. -First, it over-rides the --prefix flag specified in the configure step, -providing an alternative location. Second, it does not call hc-pkg to register the package. -Instead, third, it creates an installed package description file, installed-pkg-descr, -which can later be fed to hc-pkg. - -The reason for all this is that Roland RPM wants to create an exact installation tree, all ready -to bundle up for the target machine, but in a temporary location. He cannot use -this location for --prefix in the configure step, because that -might bake the wrong path into some compiled files. Nor does he want to register this temporary tree with the compiler -on his machine. Instead, he bundles up the temporary installation tree, plus the installed-pkg-descr, -and ships them all to the target machine. When they are installed there, the post-installation script runs -hc-pkg on the installed-pkg-descr file. - - - ---shared: if present, this flag is passed to hc-pkg -so that the package is registed as shared. This flag has no effect if --install-prefix is -used, because in that case hc-pkg is not called. - - - - - - - - -<function>setup register</function> and <function>setup unregister</function> - -The command ./Setup.lhs register registers the now-installed package with the compiler. -Similarly, ./Setup.lhs unregister un-registers the package. - ---shared: registers/un-registers a shared package as shared. -The default is to treat the package as a user package. - -We have not yet introduced the concept of a user package at this point, I think. - - - - - - -Bob the Builder and Sam Sysadmin - -Bob the Builder can install a HLI source distribution thus. -He downloads the source distribution and unpacks it into a temporary directory, -cd's to that directory, and says - - ./Setup.lhs configure --ghc - ./Setup.lhs build - ./Setup.lhs install - -Similarly, Sam Sysadmin does exactly the same, except that he says - - ./Setup.lhs install --shared - -in the final step, so that the package is installed where all users will see it. - -For a binary distribution, both Bob and Sam would omit the first two steps, and just do the install step. - - -System packagers (Debian, RPM etc) - -System packagers, such as Donald Debian, will run the configure and build steps just like Bob and Sam. -A that point, Donald will say - - ./Setup.lhs install --install-prefix=/tmp/donald - -to construct a ready-to-zip tree of all the installed files, plus a file installed-pkg-descr -that describes the installed package. He arranges to deliver both these components to the target machine, -and then feed installed-pkg-descr to hc-pkg on the target machine. - - -The file installed-pkg-descr also contains information he needs for building -his Debian distribution, namely the -package name, version, and (exact) dependencies. - - - - -HLI support for building a package: the author's viewpoint - -Angela Author has written a bunch of Haskell modules that she wants to package as a library. -What does she have to do? -She must simply build a package that can be installed as described above: nothing else is prescribed. -However, the HLI provides support for library authors to make it easy to fulfil these requirements. - - -The Makefile route - -Isaac isn't sure that we can provide much value here beside providing -a standard command-line parser (which is pretty good at least). I think it might be good to offer a boilerplate Setup.lhs file for makefiles, but implementing it in a library is a bit overkill. Perhaps in the future, if the build system is delivered w/ fptools or something, we could provide an API to wrap that. - -Marcus Makefile is happy with make, -and he can build the library entirely using make. - -To support this, HLI provides a trivial setup library Distribution.Make, which -simply parses the command line arguments and shells out into make. Marcus uses the following -Setup.lhs - - module Main where - import Distribution.Make - main = setup - -All the package description information is assumed to be known to the makefile system, and so does not -appear in the setup script. -Thus, - - setup configure --ghc - -invokes - - ./configure --with-hc=ghc - -Similarly -setup build -invokes -make all And so on. - -Marcus simply arranges that when his makefiles build a distribution, they include this simple setup script in -the root of the distribution, where the Bob the Builder expects to find it. - - -The simple build infrastructure - - -In many cases, though, a Haskell package will consist of nothing more than a bunch of Haskell modules, -with perhaps the odd C file. In that case, the HLI provides a simple build infrastructure that -completely replaces make. (Think hmake.) The emphasis is on ``simple'': it deals with the 75% case only. -If you want something more elaborate, you can (a) modify the simple build infrastructure (which is written in Haskell) -(b) use makefiles, or (c) implement something else entirely. - -The simple build infrastructure works as follows. Angela puts the following Hasell file Setup.lhs in the -root of her tree: - - module Main where - import Distribution.Simple - - pkgDescr :: PkgDescr - pkgDescr = .... - - main = setup pkgDescr - -Here PkgDescr is a Haskell data structure that describes the package: its name, version, dependencies. -It contains precisely the information that is output by setup info -- see . - -Now Angela can build her package by saying - - Setup.lhs build - -She can even install it on her own machine by saying - - Setup.lhs install - -She can build a HLI source distribution: - - Setup.lhs source-dist - -The full details are given in . - -It is no coincidence that the interface is very similar to that for the setup script -for an HLI package distribution (). -In fact, Distribution.Simple.defaultMain conforms to the specification of , and when it builds -a distribution, it includes ./Setup.lhs in the tarball, ready to be run by Bob the Builder. -However, Distribution.Simple.defaultMain of course implements a richer interface than that required by -, becuase it's intended to support Angela as well as Bob. -The full specification is in . - - -<function>PkgInfo</function> - -...describe PkgInfo... -Which compiler flags does she put in the compiler-flags part of PkgInfo, and which does she -pass to setup configure? Answer: the package should compile in a sensible way using only the -former. The latter are just for build-specific modification (e.g. make a debug build). - - - -<function>Distribution.Simple.defaultMain</function> - -Command line interface to the simple build infrastructure. - - - - - - - - -What the compilers must implement - -The HLI requires that the Haskell implementations be somewhat package-aware. -This section documents those requirements - -Building and registering a package - -Installing a package ultimately involves these steps: - - - Compiling the source files, by invoking the compiler. Even Hugs may require - some processing (e.g running cpp). - - - - Copying the compiled files into some place. Typically the compiler - places no pre-conditions on where "some place" is; instead one - usually follows the conventions of the host operating system. - - - - Registering the package: telling the compiler about the - existence of the package, and where its files are. - To register the package one invokes a compiler-specific program hc-pkg (i.e. ghc-pkg, - hugs-pkg etc), passing it an installed package description (IPD) - describing the package. - - - -It must be possible to register many versions of the same package. - -Shared packages and user packages - - -A package can be registered either as a shared package or as a user package. -The former means that anyone invoking hc will see the new package. The latter means -that only the user who installed the package will see it. - - -User packages shadow shared packages, in the following sense: - - A Haskell import for module M will seek M in a user package first. - - - The hc-pkg commands that take package IDs will look for a user package first. - - - -Hmm. If there are several ghc's around, can their user packages get mixed up? I hope not. - - - -Exposed packages and hidden packages - - -An installed package can be exposed or hidden. An exposed package -populates the module name space, while a hidden package does not. Hidden packages are nevertheless necessary. -For example, the user might use -package A-2.1 and B-1.0; -but B-1.0 might depend on A-1.9. So the latter must be installed (else B-1.0 could not be), but -should be hidden, so that user imports see A-2.1. -The registration program hc-pkg provides operations to expose or hide an -already-installed package. - - - -Registration invariants - -The registration program hc-pkg checks the following invariants: - - -Before registering a package P, check all the packages that P depends on are already registered. -If P is being registered as a shared package, P's dependencies must also be shared packages. - - -Before registering an exposed user package P, check that the modules that are exposed by P do not have the same -names (in the hierarchical module name space) as any other module in an exposed user package Q. Similarly -for system packages. (However, a system package may expose a module with the same name as a user package. - - -Before un-registering a package P, check that no package that depends on P is registered. -The exception is that when un-registering a shared package, hc-pkg cannot -check that no user has a user package depending on P. - - - - - The <function>-package</function> compiler flag - - -Registering a package means that subsequent invocations of the compiler will use a module namespace -augmented by the newly-installed package. - - -By default, the module namespace is populated only by the latest version of a particular package. -This can be overridden using the -package flag. -Question: what does -package hunit-1.2 -package hunit-1.3 do? -Does the second override the first? Or are they both in scope? I think the former. - -An attempt to import a module that is defined by more than one installed package yields an ambiguous-import -error, rather like an attempt to use an identifier that is defined by more than one imported module. -(Comment: there ought be some way to resolve such ambiguities. Per-import qualification? -disable-package X? --package Y specifies winner?) - - - -The interface to <function>X-pkg</function> - -Registering a package with a compiler records the package information in some -implementation-specific way; how it does so is not constrained by the HLI. -Much of an IPD is independent of the compiler, but it may also include compiler-specific -fields. - -Each Haskell implementation hc must provide an associated program hc-pkg which -allows a user to make a new package known to the compiler, and to ask what packages it knows. Here is a summary of its interface - -Some of these commands (unregister, hide, and describe) make sense for package IDs which offer a range, such as "hc-pkg unregister "hmake<2.3". - - <function>hc-pkg</function> interface - - - - hc-pkg register - filename - - - - Register the package using the specified installed package description. - The syntax for the latter is given in . - - - hc-pkg unregister - pkg-id - - Unregister the specified package. - - hc-pkg expose - pkg-id - - Expose the specified package. - - hc-pkg hide - pkg-id - - Hide the specified package. - - hc-pkg list - List all registered packages, both shared and user, hidden and exposed. - - - hc-pkg describe pkg-id - - Give the registered description for the specified package. - The description is returned in precisely the syntax required by - hc-pkg register. - - - hc-pkg field pkg-id - field - - - Extract the specifed field of the package description for the specified package. - - - -

-A pkg argument can be a package ID, such as "hunit-2.3", or just a package name, -such as "hunit". To determine which package is meant, hc-pkg searches first the -registered user packages and then the shared packages. If no such package exists, the command fails; that is, it does nothing, -returning a non-zero error code. -If only a name is specified, hc-pkg fails -unless the name identifies a unique package among the user packages, or among the shared pacakges. As usual, the -user packages win. - -Can we give the flag to hide, expose, -describe? Can we register a package that is already registered? What if it's registered -as a shared package and we register it as a user package? - - - -Syntax of installed package description - -...include the list of ``externally visible modules''. - - - - - -Related Systems - -I will try to outline interesting points in a variety of systems -that we can learn from. These systems may be intended for building or -installing packages, or repositories for packages. I am not deeply -familiar with all of the tools here, and would be interested in -hearing more relevant points from someone with more knowledge. -Another weakness of mine is that I don't know much about Microsoft -Windows, so some good examples for Windows systems would be -helpful. - -

Debian - - -The Debian GNU/Linux system -is a good example of a binary distribution -(meaning that packages are distributed in binary, as opposed to source -code form), and its packaging system (dpkg) -is somewhat similar to the more famous RPM. -Debian has several other tools to help the user to install packages, -most notably, apt. The Debian toolset is -interesting for several reasons: - - - - It handles dependencies extremely well. A single - command can download and install a package, as well as downloading - and installing all of its dependencies. - - It handles updates extremely well. One command - (apt-get update) checks for new versions of - packages and updates a local database. Another command - (apt-get dist-upgrade) downloads and installs all - new versions of installed packages and any new - dependencies. - - - There are standard commands for downloading and - building packages from source. If I'm interested in hacking on a - package, I can run apt-get source packagename - which will download and unpack the source code for the package. The - source can then be built with the standard command - debuild. - - - The Debian Project maintains a central repository - for packages, and the packaging tools offer support for using - unofficial repositories as well. The central repositories include a - set of servers, the autobuilders, which compile - uploaded source packages for a variety of hardware architectures - (see below) and make them available as binary packages. As a - packager, I merely upload the source code to my package, and the - autobuilders do the rest. - - Currently the hardware architectures supported by - Debian are Intel x86, Motorola 68k, Sun SPARC, Alpha, PowerPC, ARM, - MIPS, HP PA-RISC, IA-64, S/390. Debian also runs on non-Linux - systems, including GNU/Hurd, GNU/NetBSD, and GNU/FreeBSD. The - package management tools also run on MacOS X under the name of the - Fink project. - - - - -

- -

Python Distutils - -Python's &distutils; -system is in many ways similar to what we propose here. It is -a system for building and installing Python modules, written purely in -Python. The user interface is a Python script, -(setup.py by convention) and a setup -configuration file (setup.cfg by convention). To -quote from Distributing -Python Modules, "The setup configuration file is a useful -middle-ground between the setup script--which, ideally, would be -opaque to installers -- and the command-line to the setup script, -which is outside of your control and entirely up to the -installer. " - -Its noteworthy that Python has a big advantage over many -programming languages when implementing a system like &distutils;: It -is designed to be well suited to so-called scripting tasks, which are -common to the installation task, and Python has done these tasks in a -portable way for a long time. I believe that Haskell should evolve -portable ways to perform common scripting tasks. - -

- -

&cpan; and Boost - - Quoting from &cpan;'s web -site "&cpan; is the Comprehensive Perl Archive Network, a -large collection of Perl software and documentation." That really -says it all. It is a central location where Perl developers can -contribute the software they write. - -&cpan; has a means of standardizing installation, -Makefile.pl (which is a Perl script which creates -a Makefile with targets like "install", "test", "config", "clean", etc.). Makefile.pl typically uses the MakeMover -module. It also has a means of registering a namespace for the -module that a developer is contributing. - -From the Boost web -site "[Boost] provides free peer-reviewed portable C++ source -libraries. The emphasis is on libraries which work well with the C++ -Standard Library. One goal is to establish "existing practice" and -provide reference implementations so that the Boost libraries are -suitable for eventual standardization. Some of the libraries have -already been proposed for inclusion in the C++ Standards Committee's -upcoming C++ Standard Library Technical Report." - -From what I can tell, unlike &cpan;, Boost is a bit more focused -on standards and review. That is, it is perhaps more Cathedral than -Bazaar See Eric Raymond's essay The -Cathedral and the Bazaar.. Boost does not -currently have a standard means of installation. -

- -

FreeBSD's Ports System - -The FreeBSD Ports -Collection is a collection of software with a standard means -of compilation and installation. FreeBSD is a source distribution -(whereas Debian is a Binary Distribution). Packages come in -source-code form with a Makefile suitable for installing the program -on a FreeBSD system. The ports collection is very large (around 9000 -packages). - -Some things may be simpler with a source distribution than with -a binary distribution. For instance, since the code is expected to be -already on the machine and buildable, when a new compiler is installed -one merely needs to rebuild the dependant libraries. In contrast, -with a binary distribution like Debian one must wait for a new binary -package to be made available. However, as I understand it, FreeBSD -has no means of recompiling dependant packages automatically when a -new compiler is installed. - -

- - - - - -

The &xemacs; Packaging -System - - -As most folks know, &xemacs; is not only a text editor, but also a -Lisp environment. Its functionality can be extended with lisp -programs, and many such programs are available from &xemacs;' Packaging -System. Simply put, the packaging system offers a menu-driven -interface within &xemacs; where the user can browse available -packages, select packages she is interested in, and ask &xemacs; to -download and install them. This system is interesting because it is -cross-platform (Unix, Linux, Windows, etc.) and is designed to work -only with elisp. - - - - -

- -

Make-Based Systems - -The "fptools" build system has been used for many years in the -cross-platform &ghc; compiler. It is a make-based system which is -capable of a wide variety of installation tasks, compilation tasks, -and system configuration tasks. Currently, it is not entirely generic -across &impls;, and does not yet deal with some of the package -registration issues mentioned above. - -At Yale, another system is being developed. It is also a -make-based system and works reasonably well on various platforms -(Unix, Linux, Windows) and &impls;. It also does not yet deal with -all of the package registration issues mentioned above. - -Both tools can benefit from a standard packaging system. - -Because make has been used for many -years, it is expected that these systems will be able to do more than -the initial release of the &distMod;. The Setup script will be -designed with this in mind, and should be able to wrap these tools in -order to provide a common interface for users and for layered -tools. - -

- -

&hmake; - -From the &hmake; home page, -&hmake; is an intelligent compilation management tool for -Haskell programs. It automatically extracts dependencies between -source modules, and issues the appropriate compiler commands to -rebuild only those that have changed, given just the name of the -program or module that you want to build. Yes, you need never write a -Makefile again! &hmake; also does a good job of handling the -variety of compilers that might be installed on a user's system. It -maintains a list of compilers and can switch between them according to -a flag. It also has a default compiler. - -&hmake; is particularly interesting to us because it is written -in Haskell and handles the task of compiling Haskell tools quite well. -One shortcoming is that it is not extensible on a per-project basis: -it is difficult to add support for new preprocessors without editing -the &hmake; code itself. It does, however, perform a lot of the tasks -that &DistBuild; will ultimately have to perform, and we hope to reuse -some of the code. - -Another interesting feature of &hmake; is the -Haskell Interactive tool (hi). hi -is, an interpreter-like environment that you can wrap over any -common Haskell compiler to achieve an interactive development -style. This is interesting because it would be nice to have a -generic /usr/bin/haskell which would use the -default compiler to interpret Haskell scripts. - -

- - - -