HACKING

=======
HACKING
=======

:Author: Will Guaraldi
:Version: $Id: HACKING,v 1.6 2007/07/24 00:21:40 willhelm Exp $


For more up-to-date information, please check the web-site:
http://lyntin.sourceforge.net/



Notes
=====

This document provides a rough overview of how things in Lyntin are
architected and how they work.  The API documentation which is viewable
online on the Lyntin site and is in the source code itself which comes
with every copy of Lyntin (because it _is_ Lyntin) is where you will
find the answers to all your questions and more!

If you have questions that you can't find answers to, ask on the 
lyntin-devl mailing list.  Details are on the website.

In depth knowledge of the Python programming language is required to
really get into Lyntin since it's all written in Python.  If you
don't know Python, go visit http://www.python.org and familiarize
yourself with the language, its constructs. and the Python community.



Synopsis
========

Lyntin was built for the hacker mudder.  This is a special breed of
person who muds and has a series of complex problems they want to
solve that aliases and triggers aren't going solve.

Tintin has a series of commands that implements a macro language.
Building "macros" in this way is complex and solves a small set of 
problems, but the more complex problems really need a full language 
and a library to solve.  The Tintin way of solving this is to keep
adding commands that fill out the macro language to handle string
manipulations and such.  Lyntin takes a different approach leaving
those sorts of things to Python.  

Lyntin is inextricably intertwined with Python so you can
extend Lyntin to do anything without going through hoops: check your
email, record URLs you see and open up a browser to visit them, export
mudmail, building complex bots, web-servers, AIM clients, automappers....

There are a few different methods of hacking Lyntin to suit your needs.  
They are briefly outlined here and described in more detail in their 
respective sections:

1. editing the source code
2. building modules
3. command line


Before we continue, we should ramble on about Lyntin's architecture 
first and how to do things.  Before that, we'll cover Lyntin's 
philosophy.



Philosophy
==========

If you enter into the Python interactive mode (by typing "python"
at the prompt) and type "import this" you get a list of Python's
design philosophies.  When working on Lyntin, we have similar
thoughts:

* Beautiful is better than ugly.
* Simple is better than complex.
* Readability before most things.
* Even readable things need documentation.
* Optimization if we have to.
* Leave power and choice in the hands of the user.
* Get it right the third time--the first two times explore the issue 
  and perhaps implement solutions until the true solution is obvious.
* Discussion before implementation.
* Possibly specification and test cases before implementation.



Lyntin Architecture
===================

File structure
--------------

The Lyntin directory structure in CVS looks like this::

  lyntin40/
    |-- lyntin/
    |     |-- ui/       -- user interfaces
    |     |-- modules/  -- Lyntin modules
    |
    |-- scripts/        -- holds the Lyntin startup scripts
    |-- tools/          -- random stuff for testing and other stuff


When you startup Lyntin, you should specify a ``datadir``.  I store all 
my Lyntin data here (command files, session data, AIM history dumps...).



Design
------

Lyntin is multi-threaded and as such is built around an event queue.
The event queue is synchronized and allows us to have multiple threads
going without creating mutexes for everything involved.  As such, spin
things off into events whenever you can--especially when you're
interacting with the various data structures.

The engine holds the event queue and drives Lyntin.  Most Lyntin
things occur via events.  The event queue is a synchronized queue and
thus gives us a centralized way of synchronizing all the threads
accessing Lyntin internals without causing them to block.

The engine holds:

* the CommandManager
* the ConfigManager
* the HelpManager
* the HistoryManager
* all running Sessions
* the user interface
* the hook data structures
* a series of other managers which manage aliases, actions, gags, highlights, 
  substitutes, variables and other such things

Managers can manage things on an engine scoping.  For example, the 
HelpManager just has help for all of Lyntin--there's no help topics 
for a specific session.

Managers can also manage things on a session scoping.  For example,
the ActionManager (in the action plugin) manages data for each 
session.



API
---

The API for accessing Lyntin's internals is in the "exported" module.
This module contains interfaces to Lyntin's internals with the 
additional social contract that we Lyntin developers will not change
the API between minor versions of Lyntin.  Lyntin's internals actually
use the exported module for accessing its own internals.  The reasons
for this being that it gives us a centralized place for fixing behavior
bugs.

If you plan on building Lyntin modules, use the exported API--it
will save you the frustration of dealing with internal changes between
Lyntin releases.



Managers
--------

Managers manage things.  Most managers subclass the ``manager.Manager``
class.  It doesn't really provide a lot of functionality, but it 
allows us to group them all and treat them all the same.  Adding 
new managers is much easier because of this.

Also, managers register themselves with the engine via the
``exported.add_manager`` function.  The engine will cycle
through registered managers for things like status and persistence.
In addition, registered managers get told when the user has created
a new session and when the user has ended a session through the
``addSession`` and ``removeSession`` methods.

For more information, check out the ``manager`` module in the API.



Commands
--------

Lyntin comes with a series of commands for manipulating aliases, actions,
highlights, and various other things.  Both the commands and the 
managers that hold the data are all defined in various Lyntin modules
in the ``modules/`` subdirectory and are loaded up at Lyntin startup.

Lyntin has an extremely powerful argument parser that allows Lyntin
module writers to worry about the functionality of their command without
having to deal with parsing out the arguments, type-checking, 
raising exceptions with command syntax help, and data conversion.

Commands are stored internally in the ``CommandManager``.  They are
global to Lyntin--there are no commands that only exist in the session
they were declared in.  Commands, however, are executed in a specific
session.  When they are executed, they are given three arguments:
the session object for the session they were executed in, an argument
dictionary from the argument parser, and the exact input the user
typed.

Look at the ``modules/lyntincmds.py`` and ``modules/tintincmds.py`` modules 
for command examples.  Additionally, check out the Lyntin module repository
on http://lyntin.sourceforge.net/ for more examples.



Argument parser
---------------

tbd.  (Anyone who wants to write this, please do.)



Lyntin help
-----------

Lyntin has a comprehensive help system that can be accessed in-game
through the ``#help`` command.  The help material is all organized
and manipulated by the ``HelpManager``.  Help is organized as
topics which exist in a hierarchy of categories and it comes from 
a couple of sources:

1. when commands are registered via the exported.add_command
   function

2. by any module using the exported.add_help function


There has been some effort to move the ``README`` into in-game
help topics and then to actually produce the ``README`` file from
the in-game help topics.  At some point this will allow us to
create a Lyntin manual which details all the commands and their
syntaxes.



User data handling
------------------

User data (sometimes called "input") is received from the ``ui.base.BaseUI``
subclasses and tossed into an "event.InputEvent".  When the engine handles 
the event, it is first passed to the engine's ``handleUserData`` method 
which deals with global things such as splitting the input into a series 
of separate commands.  Read the docstrings for more information.

The user input is then passed to the session in question's 
``handleUserData`` method which does some stuff and passes it
through the ``user_filter_hook``.  This is where the magic happens.
All user input handlers in Lyntin (the alias manager, the variable
manager ...) register with the ``user_filter_hook`` at various 
priorities.  The ``user_filter_hook`` then cycles through the
user filters in order of priority.

For more specific information, read through the docstrings of
the methods and classes involved.



Mud data handling
-----------------

Data can also come from the mud.  Mud data comes from the net 
module's ``SocketCommunicator`` and is encapsulated in an 
``event.MudEvent``.

tbd.



The UI
------

All user interfaces extend the ``ui.base.BaseUI`` class.  It has a 
series of methods which need to be overridden to give the ui 
functionality.  In addition, ui's can specify additional Lyntin 
commands, they can bind to Lyntin hooks (most ui's bind to 
the ``startup_hook`` allowing them to perform certain functionality
at Lyntin startup rather than when the module is imported or when
the ui is instantiated), and anything else that modules can do.

Lyntin comes with a text ui which can be used over telnet/ssh
or in a command shell.  It also comes with a tk gui.  The tk gui
has functionality like split history screen (pgup and pgdown),
its own command history buffer (up and down) and numpad bindings
(``VK__NUMPAD0`` through ``VK__NUMPAD9``).

Feel free to extend the ui code and send in patches.  Or even
write your own ui based on another widget set or whatever you
want to do.

For more information, read through the code in the ui package
specifically the base module which defines the ui API.



Hooks
-----

The engine is augmented by a series of hooks which allow modules to
execute functions without having to change Lyntin's internals.  
Examples of this would be the ``startup_hook`` and ``shutdown_hook``.  
Any functions that hook into these hooks will be executed upon 
startup or shutdown of Lyntin.  Lyntin also uses these hooks to 
implement its functionality.

Plugins use hooks to hook into Lyntin and its processes.  Hooks
are defined throughout Lyntin by the various things that use the
hook.  Registering, unregistering, and retrieving Hooks should be 
done through the exported module.

Read through the API documentation for hook documentation.



Configuration
-------------

Lyntin 4.0 has a sophisticated configuration system that's composed of
boot options which are defined in the runlyntin startup script as
well as provided by the config.ini file you can pass in at the command
line.

In-game, configuration can be adjusted using the ``#config`` command.  The
``#config`` command will also tell you what configuration parameters are
available, what the values are currently set at, and descriptions of
what the configuration parameters do.

As a module writer, you can add your own configuration parameters.
See examples in the ``lyntin/modules/substitute.py`` module.

Also see the documentation in the config module in the API as well
as in the exported module.



Source code hacking
===================

Lyntin is entirely written in Python.  The source code is well organized
(this isn't really proven--it's more my opinion than anything else) 
and documentation exists on the web-site as well as in the source code 
distribution on how Lyntin is architected and details on the internals.

Lyntin is distributed under the GPL.  You should read this prior to
making changes to the source code and distributing them so you don't
accidentally find yourself in violation.

If you patch Lyntin, we'd love to see your work!  Send it to the
Lyntin developers list and depending on what the patch does, it may
get incorporated into the Lyntin base code.

Note, you must disclaim your copyright on all code you submit so we 
can assign copyright to the FSF.  If you don't, we won't accept it.



Lyntin module hacking
=====================

Lyntin modules are cool and are what gives Lyntin much of its
functionality.  Because Lyntin modules are written in Python, they 
have full access to the Python API--they're not run in any kind 
of rexec bastion or anything.

Every Lyntin module is written in Python.  Lyntin modules can be
loaded by Lyntin in two different ways.  The first is at Lyntin
startup when Lyntin lookes through all the Python modules in
the Lyntin ``modules/`` directory excluding modules whose names
start with a _.  The second is using the ``#import`` Lyntin
command::

   #import lyntinuser
   #import lyntin.modules.highlight

When the Lyntin modules is loaded, Lyntin will check the module 
for a ``load`` function and if it exists, Lyntin will execute it.  
This function lets you separate Python importing of the module 
from the Lyntin initialization.  All the Lyntin modules that 
come with Lyntin use the ``load`` function for binding to 
Lyntin hooks, adding new commands, adding new help text, and 
doing other initialization sorts of things.

When using the ``#import`` command to load a module, it first 
checks to see if the module is already loaded.  If it is and 
the module has an ``unload`` function it will call this prior to 
reloading the module and calling the ``load`` function if it 
exists.  We typically use the ``unload`` function for unregistering 
functions with Lyntin hooks as well as removing commands.  This 
allows you to cleanly reload modules you're testing/debugging 
without stopping and restarting your Lyntin session.

Lyntin modules should use the ``exported`` module to access Lyntin 
internals.  This API is guaranteed not to change (much) over versions 
of Lyntin.

In the ``modules/`` subdirectory, there's a module called modutils.  This
module holds some helper functions for making things a bit easier
to deal with.  Consult the documentation there for more guidance.

Use the modules that come with Lyntin as a series of examples on
writing your own modules.  When you're writing your first module,
be sure to brew a pot of coffee to aid you.



Writing messages to the UI
--------------------------

In your modules, you should import the ``exported`` module.  In
there are a series of ``write_`` functions which write various
Message types to the user interface.  For example::

   import exported

   def fancy_cmd(ses, args, input):

     text = args[0]

     if not text:
       exported.write_error("error: there is not text")
       return

     exported.write_message("text: %s" % text)


See the documentation in the ``exported`` module for more details.

ANSI colors are supported in the text you write to the ui.  One
way of doing this is as follows::

  exported.write_message("\033[1;34mhellothere\033[0m")


However, that's difficult to read and subject to many potential
tyops.  As such, there's a get_color function in the ``ansi`` module
which makes this a little easier::

  import ansi

  exported.write_message("%shello%s" % 
                         ansi.get_color("light blue"),
                         ansi.get_color("default"))


ANSI coloring is not a binary thing--there's no end tag.  It's
also not very nestable.  Make sure to use ``default`` when ending
your ANSI coloring--that returns the ANSI color to the default
values for options, foreground, and background.



Command line hacking
====================

Lyntin allows you to execute arbitrary Python at the command line.
If you have a ``lyntinuser`` Lyntin module, then the code will be executed 
in there.  Otherwise the code is executed in the ``advanced`` Lyntin 
module.

For example::

   #@print "hello"

will print hello to the console.  There are some significant differences
between Lyntin 2.x and 3.x in how this works.  In the 2.x series, 
raw Python was executed in the ``user`` module.  It was encouraged that 
you add your own functions to the ``user`` module and then these would
be available to you at the Lyntin command line.

In 3.0 and later versions, ``lyntinuser`` is a Lyntin module in the 
modules directory.  It does not come with Lyntin--it's something 
you can create if you want to build your own environment to execute 
Python commands in.

If you don't have a ``lyntinuser`` module,  Raw Python code gets executed in 
the ``advanced`` module where the #@ functionality exists.

A basic "lyntinuser" module could be as follows::

   """
   This is my lyntinuser.py module.  I can do whatever I want in here.
   I can store this module in my Lyntin moduledir (set at the command line
   using --moduledir <dir>) or in Lyntin's modules subdirectory along
   with alias.py, action.py and the rest of the gang.
   """
   import exported
   myvar = "hello"

   def myfunc():
     exported.write_message(myvar)
   
   def load():
     """
     You can put code here for initializing your module when it's 
     loaded here.  Typically this is registering with Lyntin hooks
     and adding Lyntin commands.  Look at the other Lyntin modules 
     for examples.
     """
     pass

   def unload():
     """
     Put code here for uninitializing your module when it's unloaded.
     Typically this is the reverse of the load module: unregistering
     functions with Lyntin hooks, remove Lyntin commands, removing
     Lyntin help topics...  Look at other Lyntin modules for examples.
     """
     pass


Now at the Lyntin command prompt I can do things like::

   > #@print myvar
   hello
   > #@myfunc()
   lyntin: hello



The end
=======

That about covers it.  Read through the API and the source code.  If you 
have questions, subscribe to the lyntin-devl mailing list and ask us.  

http://sourceforge.net/mail/?group_id=6730


Happy hacking!

the Lyntin development folks
http://lyntin.sourceforge.net/