RedtamarinPackaging

Redtamarin Packaging

First, consider all this as a work in progress, as long as our major version is zero it could change anytime.

Here we try to define rules and policy on how to package Redtamarin, but also command-line tools, shell scripts, ActionScript 3.0 libraries etc.

Problem

It's basically about distribution, installation, and organisation.

Stuff like: Where to put things, How to name things, How to do it in a cross-platform way, etc.

Here few scenarios.

Installing Redtamarin

know which version is installed
in which path to install it
how to access it on the command-line ?
what do we install exactly ?
- only the redshell for the current system?
- the redbean and other tools ?
- common ABC libraries ?
- what about the documentation ? only man? html?
- how to upgrade from version x.y to x.y+1 ?
- can we have different versions in parallel ?
- what about if a library depend on a specific version ?

Installing Redtamarin Libraries

where to install and share them ?
can we have private libraries ?
can we install different versions of the same library ?
ex: appA using httplib-1.0 and appB using httplib-1.1

and many many more questions like that ...

To solve all those problems, we will study what other programming languages have been doing for many years.

Even if we study: Perl, PHP, Python, Java, Node.js, Erlang, etc., we are not exactly like them, so we need to learn and find our own way.

We can learn a lot by studying Linux FHS and Debian Packaging Policies

We can learn a bit more from great projects like

pandoc
curl
systemd
and many others

Cross-Platform

For now, we will follow the Debian packaging format to install and distribute

Redtamarin for Windows
Redtamarin for Mac OS X
Redtamarin for Linux

Reasons:

Windows sucks at command-line installation/distribution
but it is not too hard to make it compatible with a sane POSIX environment
see Redtamarin Windows Environment Setup
Mac OS X inherits a POSIX environment from its BSD origin
the native way to install command-lines tools is to build a .pkg
which is very similar to build a .deb
Linux has a POSIX environment by default
.deb will works by default on Debian and Ubuntu
we need a way to package our different executables, config, documentation, etc.
the .deb format is convenient and practical for that
it's close to a zip file but with the added advantage to use "control" files
even if we had our own package management tool
we would still need a way to install Redtamarin first

Redtamarin Packages

Here the list of different executables and utilities we need to install

redshell
the Redtamarin Release Shell
redshell_d
the Redtamarin Debug Shell
redshell_dd
the Redtamarin Debugger Shell
we can also refer to the shells as
"The Redtamarin Runtimes" when we talk
about: redshell, redshell_d, and redshell_dd
redbean
the Redtamarin build tool
asc.jar / asc2.jar
ActionScript Compiler
as3run
bash script to run an .as file with binfmt_misc
abcrun
bash script to run an .abc file with binfmt_misc
swfrun
bash script to run a .swf file with binfmt_misc

Optional tools:

as3shebang
the ActionScript 3.0 Interpreter
other tools based on redtamarin

Because we depend on a POSIX system and want to distrib Debian packages we will follow the Filesystem Hierarchy Standard.

Our basic organisation should be

executables in /usr/bin/
for ex: /usr/bin/redtamarin
libraries in /usr/lib/redtamarin/
for ex: /usr/lib/redtamarin/lib
command-line tools manual in /usr/share/man/man1/
for ex: /usr/share/man/man1/redtamarin.1.gz
documentation files in /usr/share/doc/
for ex: /usr/share/doc/redtamarin/

/usr/lib/redtamarin should contain all our files and if we need to deal with different redtamarin version we can simply manage them with symlink

redtamarin -> redtamarin-0.4.5
redtamarin-0.4.1
redtamarin-0.4.2
redtamarin-0.4.3
redtamarin-0.4.4
redtamarin-0.4.5

File Structure

structure of /usr/lib/redtamarin

/usr/lib/redtamarin
               |_ bin                <-- contains only executables considered part of redtamarin
               |    |_ redtamarin    <-- our main executable
               |    |_ redshell      <-- our release runtime
               |    |_ redshell_d    <-- our debug runtime
               |    |_ redshell_dd   <-- our debug-debugger runtime
               |    |_ redbean       <-- our build tool
               |
               |_ support            <-- to add any external executables/modules/libraries
               |    |_ asc.jar           "external" as not part of redtamarin
               |    |_ asc2.jar
               |
               |_ lib                <-- to add ABC/SWC public libraries (non-executable)
               |    |_ redtamarin.abc
               |    |_ redtamarin.swc
               |    |_ redtamarin.txt
               |
               |_ run                <-- to add any ABC/SWC/Shell Scripts (executable)
               |    |_ redbean.abc
               |
               |_ runtimes           <-- to add runtimes by NAME/PLATFORM/BITS/exec_name
                    |_ redshell
                          |_ windows
                          |      |_ 32
                          |      |   |_ redshell.exe
                          |      |   |_ redshell_d.exe
                          |      |   |_ redshell_dd.exe
                          |      |
                          |      |_ 64
                          |          |_ redshell.exe
                          |          |_ redshell_d.exe
                          |          |_ redshell_dd.exe
                          |
                          |_ macintosh
                          |      |_ 32
                          |      |   |_ redshell
                          |      |   |_ redshell_d
                          |      |   |_ redshell_dd
                          |      |
                          |      |_ 64
                          |          |_ redshell
                          |          |_ redshell_d
                          |          |_ redshell_dd
                          |
                          |_ linux
                                 |_ 32
                                 |   |_ redshell
                                 |   |_ redshell_d
                                 |   |_ redshell_dd
                                 |
                                 |_ 64
                                     |_ redshell
                                     |_ redshell_d
                                     |_ redshell_dd

/usr/lib/redtamarin/bin
should contains all the redtamarin executables for the current operating system

ex: if you are on a Linux 64-bit
and so have installed redtamarin_0.4.1_amd64.deb
then you should have installed by default al the redshell for Linux 64-bit

Every Redtamarin binary executables installed in /usr/bin
should be a symlink from /usr/lib/redtamarin/bin

ex:
/usr/bin/redshell -> /usr/lib/redtamarin/bin/redshell

But small shell scripts like abcrun should be directly
installed in /usr/bin
eg. /usr/bin/abcrun

#!/bin/sh

ABCNAME=$1
shift
/usr/bin/redshell "${ABCNAME}" -- $@

/usr/lib/redtamarin/lib
should contain all public non-executable library accessible to any programs
by default it will contain

redtamarin.abc
required to compile programs to .abc
redbeanlib.abc
utility library
other ?
other libraries could be included by default

note:
the following are different things

/usr/lib/redtamarin/lib/redbeanlib.abc
is a non-executable pre-compiled library
running $ ./redshell_dd redbeanlib.abc will do nothing
/usr/lib/redtamarin/run/redbean.abc
is a pre-compiled library that can be executed
running $ ./redshell_dd redbean.abc -- -f test.as3 is the equivalent of
running $ ./redtamarin --debugger redbean.abc -f test.as3 is the equivalent of
running $ ./redbean -f test.as3
/usr/lib/redtamarin/bin/redbean
is a projector that combines redshell_dd and redbean.abc
into a standalone executable

/usr/lib/redtamarin/run
should contain all public executable library
by default it will contain

redbean.abc
other?
maybe

note:
A library that can be executed is like a shell script
the only difference is that it is a binary blob, not plain text.

The binary is executable but does not contain the Redtamarin runtime.

The binary can be either an .abc or a .swf file.

You need to use either one of the redshell or the redtamarin utility
ex:
$ redshell myprogram.abc
$ redtamarin myprogram.abc
$ redshell myprogram.swf
$ redtamarin myprogram.swf

Under Linux, thanks to binfmt_misc, you can directly run those
ex: $ myprogram.abc, $ myprogram.swf

At the time of packaging, all public library should be
prefixed by as3- (this could change)
format: as3-<library-name>

Let's say you have a library named httplib
you will package it as as3-httplib_1.0_all.deb

prefix as3-
we picked up as3- because it reflect
better the different kind of libraries we can package
AS3 compiled to ABC, but can also contains a .swc
or even AS3 sources to be compiled
the name of the library
should be unique and we should have some kind
of registry or repository somewhere
the version of the library
basic semantic version with patch optional
all to indicate it will works
on all architectures

The fact that the name of the library end with lib
is purely optional and only a convention we plan to
adopt for our own libraries.

In /usr/lib/redtamarin/lib/ will not do versioning

one library
for ex: httplib
install at least one .abc file
ex: httplib.abc
can install also one .swc file
ex: httplib.swc
can install one version .txt file
ex: httplib.txt will contain 1.0

The version .txt file can also be generated by redbean
for ex: $ redbean version httplib.abc
will read httplib.abc, obtain its version
and produce the text file httplib.txt
in the same path as the .abc lib

Public libraries can be loaded from different places

load( "httplib.abc" )
first, will look in the current directory
second, will look in all the paths defined
by the environment variable REDTAMARIN_PATH
for ex: export REDTAMARIN_PATH=/home/zwetan/abclibs/:/usr/share/redtamarin/experimental/
third and finally, will look in
/usr/lib/redtamarin/lib/

When it comes to executable libraries we treat them differently
than non-executable libraries

the name should not be prefixed by as3-
but the arch should be all
also the library could auto-generate
a standalone executable

Here 2 examples:

Distribute the redbean projector

package as redbean_1.0_amd64.deb
can only be installed for 1 operating system
install in /usr/share/<package-name>/
but here redbean is an exception
as it is distributed with redtamarin package
so we install it in /usr/lib/redtamarin/bin/
will create symlink to the executable
in /usr/bin

Distribute the redbean.abc executable library

package as redbean_1.0_all.deb
can be installed for all operating systems
supported by redtamarin
install in /usr/lib/redtamarin/run/
will create a bash script in /usr/bin
for ex: /usr/bin/redbean

#!/bin/sh
/usr/bin/redshell "/usr/lib/redtamarin/run/redbean.abc" -- $@

or

#!/bin/sh
/usr/bin/redshell_dd "/usr/lib/redtamarin/run/redbean.abc" -- $@

or

#!/bin/sh
/usr/bin/redtamarin "/usr/lib/redtamarin/run/redbean.abc" $@

etc.

Documentation

Redtamarin can have different type of documentations

for command-line tools
- the classic man page
- but also HTML
for the Redtamarin API
shared by redtamarin, redshell, redbean, as3shebang, etc.
misc. files related to
changelog, release, copyright, license, etc.

Each command line tools will have a man page

for redtamarin
see /usr/share/man/man1/redtamarin.1.gz
for redbean
see /usr/share/man/man1/redbean.1.gz
for as3shebang
see /usr/share/man/man1/as3shebang.1.gz
etc.

The Redtamarin API HTML documentation will be shared here
/usr/share/doc/redtamarin/api/

By extension any other tools or libraries generating HTML doc from asdoc should also use the api directory
for ex: /usr/share/doc/redbean/api/, /usr/share/doc/as3-httplib/api/, etc.

Other tools and/or libraries HTML documentation will follow the following
/usr/share/doc/$NAME/html/

This is to document the tool itself, for example: usages, examples, etc.

for example:

redbean HTML documentation
see /usr/share/doc/redbean/html/ how to use the redbean tool
see /usr/share/doc/redbean/api/ HTML doc generated by asdoc
as3shebang HTML documentation
see /usr/share/doc/as3shebang/html/ how to use the tool to create AS3 shell scripts and other usages
but no /usr/share/doc/as3shebang/api/ as there is no HTML generated by asdoc
as3-httplib HTML documentation
see /usr/share/doc/as3-httplib/html/ description and usage of what you can do with the library
see /usr/share/doc/as3-httplib/api/ for the HTML doc generated by asdoc
etc.

With a special case for redtamarin

/usr/share/doc/redtamarin/api/
the Redtamarin API HTML documentation
/usr/share/doc/redtamarin/html/
the Redtamarin command-line tool HTML documentation

For misc. files like changelog, readme, release notes, etc.
we will use the following /usr/share/doc/$NAME/

Here some file structures to show how it would look like

redtamarin

/usr/share/doc/redtamarin
                   |_ api
                   |    |_ index.html (asdoc)
                   |    |_ ...
                   |
                   |_ html
                   |     |_ redtamarin.html (usage, example, etc.)
                   |     |_ ...
                   |
                   |_ README
                   |_ CHANGELOG
                   |_ RELEASE
                   |_ etc.

As a general manner

we will use asdoc to generated HTML documentation
for AS3 API, libraries, etc.
we will use markdown and pandoc to generate HTML documentation
for description, usage, examples, etc.
other files like readme, changelog, release, etc.
will use the markdown notation

Package Naming for Distribution

We have numerous elements in different places: executables, documentation, libraries, etc.

Here what goes into which package and with which name.

redtamarin_x.y.z_arch.deb should contain only

the redtamarin executable
the redtamarin.1 man
if it can not found the redshell executables
will show an error
the redtamarin.abc
the redtamarin.swc

redtamarin-doc_x.y.z_arch.deb should contain only

the generated HTML documentation from asdoc
HAVE TO have the same version as the redtamarin package

about redshell packages:

the arch is always all
because here it represent "which system can install it"
not "which system can execute it"
because we use those redshells only to package projectors
we can distribute on all systems, and so
distribute the same packaged .deb
the redshells being installed in /usr/lib/redtamarin/bin
should be a copy or symlink from /usr/lib/redtamarin/runtimes/redshell/linux/64
which "linux" and "64" detected at install time

redshell-windows_x.y.z_all.deb should contain only

Windows executables
both 32-bit and 64-bit
HAVE TO have the same version as the redtamarin package
the arch is always all

redshell-macintosh_x.y.z_all.deb should contain only

Mac OS X executables
both 32-bit and 64-bit
HAVE TO have the same version as the redtamarin package
the arch is always all

redshell-linux_x.y.z_all.deb should contain only

Linux executables
both 32-bit and 64-bit
HAVE TO have the same version as the redtamarin package
the arch is always all

redbean_x.y.z_arch.deb should contain only

the redbean executable
the redbean.1 man
if it can not found some path or env vars
will show an error
the redbean.abc in /usr/lib/redtamarin/run
the redbeanlib.abc in /usr/lib/redtamarin/lib
the redbeanlib.swc in /usr/lib/redtamarin/lib
SHOULD contain asc.jar and asc2.jar
and install them in /usr/lib/redtamarin/support

redbean-doc_x.y.z_arch.deb should contain only

the generated HTML documentation from asdoc
HAVE TO have the same version as the redbean package

redtamarin-sdk_x.y.z_arch.deb contains all of the above

basically the main reason we name it a SDK
we need numerous runtimes, libraries, build tool, compilers tool, etc.
all that is complex
as long as we are in major version 0
to keep things simpler we should only
install 1 package that contains all those
different packages
the main problem with that is the size of the package
which is acceptable for a "SDK"
force us to have a post-install script as

#!/bin/sh  
cp /usr/lib/redtamarin/runtimes/redshell/linux/64/redshell_dd /usr/lib/redtamarin/bin/

or

#!/bin/sh  
$PLATFORM # detect platform
$BITS     # detect 32 or 64 bits 
cp /usr/lib/redtamarin/runtimes/redshell/$PLATFORM/$BITS/redshell_dd /usr/lib/redtamarin/bin/

or

#!/bin/sh  
UNAME="$(uname)"
BITS="32"
EXT=""

# detect platform
if [ $UNAME == "Darwin" ]; then
    PLATFORM="macintosh";
elif [ $UNAME == "Linux" ]; then
    PLATFORM="linux"
elif [ "$(uname -o)" == "Cygwin" ]; then
    PLATFORM="windows"
    EXT=".exe"
fi

# default is 32-bit, detect 64-bit
if [ "$(uname -m)" == "x86_64" ]; then
    BITS="64"
fi

# copy the 3 redshells to bin
cp /usr/lib/redtamarin/runtimes/redshell/$PLATFORM/$BITS/redshell    /usr/lib/redtamarin/bin/
cp /usr/lib/redtamarin/runtimes/redshell/$PLATFORM/$BITS/redshell_d  /usr/lib/redtamarin/bin/
cp /usr/lib/redtamarin/runtimes/redshell/$PLATFORM/$BITS/redshell_dd /usr/lib/redtamarin/bin/

Current

From now on, since v0.4.1 we will distribute
"The Redtamarin Software Development Kit"
redtamarin-sdk_x.y.z_arch.deb which contains

"The Redtamarin command-line utility"
redtamarin_x.y.z_arch.deb
"The Redtamarin HTML Documentation"
redtamarin-doc_x.y.z_all.deb
"The Redtamarin Runtimes for Windows"
redshell-windows_x.y.z_all.deb
"The Redtamarin Runtimes for Mac OS X"
redshell-macintosh_x.y.z_all.deb
"The Redtamarin Runtimes for Linux"
redshell-linux_x.y.z_all.deb
"The Redtamarin Build Tool"
redbean_x.y.z_arch.deb
"The Redtamarin Build Tool HTML Documentation"
redbean-doc_x.y.z_arch.deb

note:

Windows installation depends on
redtamarin-setup.bat
see Redtamarin Windows Environment Setup
"The ActionScript 3.0 Interpreter"
as3shebang_x.y.z_arch.deb
is a separate and independent package on purpose

Native Packaging

In the future, we should be able to distribute also

redtamarin-sdk_x.y.z_arch.pkg
native package installer for Mac OS X
redtamarin-sdk_x.y.z_arch.msi
native package installer for Windows

Provide feedback

Saved searches

Use saved searches to filter your results more quickly