Twilight

Getting Started with Twilight:

Twilight presently supports Microsoft
Windows XP, Mac OS/X 10.4, and most any contemporary Linux version.
Porting to any Posix-compliant, Unix like platform which the
Hans-Boehm collector supports and which supports the gcc should be
trivial, please see the Cross Generation and porting information
below for instructions. For the Windows port it is necessary to have
a recent version of Visual C++ installed, I have successfully used
the free 2005 Express edition. At some point in the future mingw or
another GCC port to Windows may be supported as well. Mac OS builds
are currently being done on Darwin 8 on pearpc, it is the download
from Apple with their GCC compiler so I expect it will work fine on
10.4 but I have not been able to confirm it. It is possible to
cross-generate from any existing binary installation to any other and
then build on the target platform. Over time it may be possible to
add cross-compilation capabilities as well.

Insure that the development environment
consisting of GCC and make on Posix platforms, MS VC++ (incuding the
free Express version) and the Microsoft platform SDK on Windows are
installed and working.

Platform Specific Notes:
On Linux, most distributions come with
gcc already installed. If it is not (execute "gcc" at the
command line, if it is not found, it is probably not installed), you
will need to refer to your distribution specific documentation for
how to install.

On MacOS you will need to have the
Apple GCC installed, it is available as part of Xcode, available from
Apple at http://developer.apple.com/tools/xcode/

On Windows you will need an MS VC++
compiler and platform SDK installed, install the compiler first then
the SDK, when installing the SDK be sure to opt to integrate it with
the VC++ installation when it prompts during the install process.
Free downloads for these pieces of software are available at:
http://msdn.microsoft.com/vstudio/express/visualC/default.aspx
and
http://www.microsoft.com/downloads/details.aspx?FamilyID=eba0128f-a770-45f1-86f3-7ab010b398a3&DisplayLang=en
respectively. Once installed, you will need to open the command shell
for the SDK (Windows XP Release...) which includes the pathing for
both the compiler and the SDK,

Change directory to twhome/tw/current
directory extracted from the binary download, source the appropriate
platform specific file (execute the .bat file on Windows) , and you
should be ready to go. You will need to remain in that directory
during all build executions.

View the Examples/HelloWorld.tw file
and execute the following command line to build it:

TWX_StdLib.exe -buildFile
Examples/buildFiles/helloWorld.txt

It will generate, compile, build, and
execute the program, which should result in "Hello World"
output to the console.

In normal use you will have a buildFile
and that will be the only argument for the command line. More detail
regarding the build process is available below, to become familiar
with code and the library, first look at the *.tw files under Tests
for small working programs which exercise functionality. Work with
those, write your own, etc, when you are feeling adventurous look in
StdLib for the standard library files, and when you feel up for even
more, have a look at the generator itself, contained in Gen/*.tw

All of the arguments to the build are
on the command line for the generator, which is built to a file named
TWX_StdLib.exe on each platform. Additional arguments can be
specified with -buildFile FILE.txt arguments, the FILE.txt argument
will need to be a path to a file containing argument entries and
values for those (where appropriate) delimited by newline just as
they could have been defined on the command line space-delimited. In
other words, the snippit (this is not a full working command line...)

TWX_StdLib.exe -deployPackage
deployTest/HelloWorld -buildPath buildTest/HelloWorld -compileUnit
StdLibTest_HelloWorld -main Hello:World HelloWorld.tw

is equivalent to

TWX_StdLib.exe -buildFile
buildFiles/buildTest.txt

where the file at the path
buildFiles/builtTest.txt contains the following:

-deployPackage
deployTest/HelloWorld
-buildPath
buildTest/HelloWorld
-compileUnit
StdLibTest_HelloWorld
-main
Hello:World
HelloWorld.tw

You can have multiple -buildFile
references on the command line and the arguments contained within
them (newline delimited) will be read just as if they were all
appended to the command line space-delimited. A buildFile can refer
to other buildFiles as well. This mechanism is used to make the
overall build system multi-platform while containing platform
specific bits in a limited set of buildfiles which are copied into
place on each platform - specifically, in the
twhome/tw/current/buildFiles directory, the files buildPlatform.txt
and buildPlatformStdLib.txt

There is no hard-coded relationship for
the locations of the buildFiles, etc, or the relative pathing found
throughout the build area - it is a convention which has built up as
a convenient mechanism which allows for relocation, at the expense of
needing to be in a particular directory when the build command is
invoked.

Each build process defines a
compilation unit which must be uniquely named amongst all which are
incorporated in to a running program. The build process creates a
work directory defined by the -buildPath argument, this directory
contains timestamp files, generated .c and .h files, the object files
during compilation, .syn files which provide class information
required by the generator during builds, and files listing all of the
names (members and methods) which are present in the compilation
unit. Builds are incremental and generally do a good job of managing
dependencies, if in doubt you can remove the buildPath directory to
force a complete re-generation and compilation. Every compilation
unit requires a Main class which is the entry point to the
executable, if a compilation unit is meant to be used as a library
the main method may do very little, the class System:DoNothing is
provided in the standard library and can be used in this case if
desired. The product of the build is a package deploy directory
structure which contains C headers, .syn files, and a dynamic library
and linked executable which provide entry points for re-use as a
library and the starting point for program execution respectively.
The deploy package area can be used from other compilation units with
the -usePackage directive. For reasons related to the problem of
working on a self-hosting system the generator includes the complete
standard library and can operate as a stand-alone entity. This could
be done with any applicaction, of course, but
generally I expect it will not be so as to realize the efficiencies
of shared code

. The library build and test packages
(see Gen/buildFiles/buildlib.txt and Tests/buildFiles/*.txt
respectively) are the better model for development outside the
generator itself. There are actually several generations of compiler
build areas, buildtw3, buildtw2, and buildtw0, again related to the
process of building the self-hosting generator and not for general
development tasks.

Most of the build files refer to files
outside themselves so as to reduce duplication and allow for
cross-platform builds. The cross-generation build files cannot do
that because they do not match the normal build so they include all
arguments required for a build without external references. The
cross-generation build files are in twhome/tw/current/Gen/buildFiles
along with the other standard build files, they are named
build0mc.txt, build0wc.txt, build0lc.txt for generating to Macos,
Microsoft Windows, and Linux respectively. They are a good place to
look to see all arguments arequired for a build.

Here is the example cross-generating
build file for Linux. Explanatory comments are above actual entries
like so <Explanation>. Note that this is not a mechanism for
commenting in the buildFiles and should not be incorporated into them
at present.

<gen only is for cross-generating
builds only, this insures that the compile of source to native code
and execution of code does not occur>
-genOnly
<defines the platform for the build
which will control aspects of generation, makefile creation, and
compiler syntax>
-platform
linux
<a file which is part of what is to
be compiled, out of place with the remaining files to be parsed due
to it's platform specific nature>
StdLib/FileNixish.tw
<print visitors for diagnostic
purposes>
-printVisitors
<the include path for the base
TWR_Base.h and .c files which define some basic structures and
methods>
-includePath
includetw
<an external library entry ,
multiples allowed, this one is for the garbage collector library.>
-extLib
../../lib/linux/gc/lib/libgc.so
<an external include area for during
the build, in this case the include for the garbage collector>
-extInclude
../../lib/linux/gc/include
<the path to deploy the package to>
-deployPackage
deploy0
<the build path for temporary files
and timestamps related to the build>
-buildPath
buildtw0
<the name of the compilation unit,
must be StdLib for the standard library build, must be unique amongst
all -usePackage entries for a given program>
-compileUnit
StdLib
<the name of the main class, a
no-argument contsructor constructs and instance which is then
invoiked with the "main()" method.>
-main
Gen:Build
<the name of files to be included in
the build are below>
<this is the Gen area which is
specific to the parser-generator>
Gen/Transport.tw
Gen/Package.tw
Gen/Build.tw
Gen/Nodes.tw
Gen/EmitData.tw
Gen/Errors.tw
Gen/Constants.tw
Gen/NodeTypes.tw
Gen/CEmitter.tw
Gen/BuildTypes.tw
Gen/InlineTypes.tw
Gen/Syns.tw
Gen/Visitor.tw
Gen/Pass1.tw
Gen/Pass2.tw
Gen/Pass3.tw
Gen/Pass4.tw
Gen/Pass5.tw
Gen/Pass6.tw
Gen/Pass7.tw
Gen/Pass8.tw
Gen/Pass9.tw
Gen/Pass10.tw
Gen/Pass11.tw
Gen/Pass12.tw
Gen/CEmitVisit.tw
<this is the standard library file
set, incorporated into the parser-generator directly but not for most
development efforts>
StdLib/Logic.tw
StdLib/LinkedList.tw
StdLib/Stack.tw
StdLib/Array.tw
StdLib/Vector.tw
StdLib/Logic.tw
StdLib/Map.tw
StdLib/Math.tw
StdLib/System.tw
StdLib/Object.tw
StdLib/Exceptions.tw
StdLib/Text.tw
StdLib/StringBuilder.tw
StdLib/Tokenize.tw
StdLib/FilePath.tw
StdLib/FileReadWrite.tw
StdLib/Process.tw
<the end>

In the twhome/tw/current area are
setenv_platform files to source (or on Microsoft Windows, execute)
from the command shell before performing builds. It is not mandatory
for Nixish (LiNix like(ish)) operating systems, *BSD, macos, linux,
proprietary Unix flavors as they become supported, but is for
Microsoft Windows due to how dynamic libraries are handled on that
platform.

In order to add functinality to the
standard library, if you are introducting a new file add them to the
.txt files within Gen/buildFiles/*.txt and build. Tests for
functionality belong under Tests, there are .tw files for code and
.txt files within Tests/buildFiles for tests classes. Have a look at
them for examples of how to link packages together, these refer
generally only to the StdLib package, but as many packages as you
wish can be linked together.

The standard process:

If changes or additions were made to
StdLib,

TWX_StdLib.exe -buildFile
Gen/buildFiles/buildlib.txt

then (or only, if no additions made to
StdLib):

TWX_StdLib.exe -buildFile
Path/To/Your/buildfile.txt

(if it is a test,
Tests/buildFiles/yourClass.txt)

At the least, buildfile.txt should
contain:

-buildFile

buildFiles/buildPlatform.txt
-usePackage
deploy
-run
-deployPackage
deployWork/YourClass
-buildPath
buildWork/YourClass
-compileUnit
YourOriganization_YourClass
-main YourOrganization:YourClass
Work/YourClass.tw

and Tests...yourClass.txt

-buildFile
buildFiles/buildTest.txt
-deployPackage
deployTest/YourClass
-buildPath
buildTest/YourClass
-compileUnit
StdLibTest_YourClass
-main Test:YourClass
Tests/YourClass.tw

Extract the archive for your platform a
directory of your choice. It will extract to a top-level directory
twhome. Open a command line and change to twhome/tw/current. On
MSWindows, execute setenv_mswin.bat, on other platoforms source the
setenv_(platform).sh file. Create and edit .tw files contianing your
code and buildFiles/*.txt files as described above, execute the above
commands to perform your build. Most syntax errors should throw
exceptions during the build as will type errors for statically typed
code, errors related to dynamically typed references will appear at
runtime. The -run argument cause the resulting executable to be
invoked after build. The executable and library will also be in your
deploy directory as defined with the build argument -deployPackage
(directory), deployDir/emit/c/CompileUnit/. In the case of microsoft
windows it must be on the path, the easiest way to accomplish that is
to copy the contents of the CompileUnit directory into
twhome/tw/current/bin Regardless of the location of the executables
they must be executed from within the directory twhome/tw/current due
to the paths as they are incorporated into the buildfiles.

Deployment of code:

The deploy directories which are
defined with the -deployPackage argument contain everything necessary
to deploy code either for execution or incorporation into other
programs. They include "c" header files and Twilight .syn
and name files as well as binary dynamic library and executable. If
you incorporate other packages into your package with -usePackage
directives, those packages directories, along with any they may use,
must be included in your deployment process as well. Whatever
mechanism is required to place shared libraries on the library search
path may be required if the libraries are relocated relative to one
another. In the case of Microsoft Windows it is best to have all
(packagedir)/emit/c/CompileUnit directories within the PATH
environment variable or copied into a directory which appears on the
path. For MacOs and Linux, if relative paths are maintained between
packages, from the directory just above the deploy directory you can
execute the executable and all libraries should be found properly,
failing that the LD_LIBRARY_PATH can be modified to include all of
the (packagedir)/emit/c/CompileUnit directories for packages which
you use.