FUNCTION: This program is an elevator simulator written by James
Lewis, Bob Crites, and John McNulty, and was provided by Professor
Christos Cassandras of the ECE department at UMASS.  Please cite them for
providing this code.

REQUIREMENTS:
The simulator is in FORTRAN, but includes random number handling routines
written in C.  As there is a history of trouble with compiling the
FORTRAN routines on Sun Sparc architectures, you *may* need the f2c
libraries, available at: http://www.netlib.org/f2c/

INSTALLATION
To install:
1) If you do not yet have the f2c program installed, download it from
http://www.netlib.org/f2c, and install it.  This will involve compiling
the source code into a single f2c library.  You will need to know
the location of this library, and the location of the f2c include files,
for the next step of this installation.

2) Edit Makefile, and change the line:
	CFLAGS =
   to include the location of the f2c include files.

   Also, change the line which begins:
        LIBS =
   to include the location of the f2c library file.

3) Also, on many systems, you will need to set an enviroment variable so
that the linker can find these files.  An example is:
 	setenv LD_LIBRARY_PATH /user/local/lib/f2c:
		/user/local/lib/f2c/libF77:/user/local/lib/f2c/libI77

     (Note: this should be all on one line.  It was split into two here
      for reading convenience.)

4) Type "make" to compile.

agent.f and net.f implement the reinforcement learning components.  

***********************************************************************
INPUT:  There are three relevant input files, input.dat, fort.1,
and parms.dat.


(1) file "input.dat"
      First section:
	1,ngraphics - whether to display simulation graphics (0=no; 1=yes)
	60 , max_wait
	0, nstart
	0 , nprint
	0 , nalg
	0 , debug - whether to display debugging info (0=no; 1=yes)
	4 , cars
	20 , capacity
	10 , floors
	1 , ground floor            
	1.45 , floor time
	1.0 , enter time
	1.0 , exit time
	7.19 , stop time                                  
	1 ,0 ,1 ,0  ,1 ,0 ,1 ,0 ,1 ,0 ,direction
	1 ,5 ,6 ,10 ,9 ,8 ,6 ,4 ,2 ,1 ,floor
	1 ,5 ,6 ,10 ,1 ,10 ,1 ,10 ,1  ,10 ,park         

The following lines represent the passenger arrival process.  The
first line is for floor 1, the second line for floor 2, etc.  The
extra lines at the end are ignored.  The first number on each line is
a scaling percentage.  By setting it to zero, one would turn off all
arrivals to that floor.  The next 12 numbers are the mean number of
passengers to arrive during each 5-minute interval.  The remaining
numbers are ignored.

200,1,1,1,1,1,1,1,1,1,1,1,1,10,10,10,10,10,10,10,10,10,10,10,10,uni
100,1,2,4,4,18,12,8,7,18,5,3,2,5,5,5,5,5,5,5,5,5,5,5,5,uni
100,1,2,4,4,18,12,8,7,18,5,3,2,5,5,5,5,5,5,5,5,5,5,5,5,uni
100,1,2,4,4,18,12,8,7,18,5,3,2,5,5,5,5,5,5,5,5,5,5,5,5,uni
 .
 .
 .

Notes on passenger arrival parameters:

a.) The first number on every line is a scaling number. 100 means do
not scale. 200 means double if there is no inter-floor traffic.

b.) FLOOR 1 is handled differently from the other floors!  The numbers
following the first number specify the mean number of passengers
arriving PER 30 seconds for each 5 minute interval. For example, the
following line says: 

100,2,2,2,2,2,2,2,2,2,2,2,2,10,10,10,10,10,10,10,10,10,10,10,10,uni

"Don't scale, and there are 4 passengers arriving per minute for each 5
minute interval for the 1st hour". 

"For the second hour, there are 20 passengers arriving per minute for
each 5 minute interval". (GASP ERROR!)

c.) For every floor other than the 1st floor, the numbers following the
first number specify the mean number of passengers arriving every 5
minutes. 


(2) FILE "fort.1":

    The first line looks like this:

	GEN,PRGM,1,05,16,1996,2,1,N,N,Y,Y,Y*

    Fields 4-6 represent the date.
    Field 7 is how many hours to simulate.
    Fields 11-13 tell whether to print out lots of statistical info.

    Note that this specifies how many 1 hour runs to do, so this example
    means do 2 1 hour runs. (The 1 following the 2 is undetermined!)

(3) parms.dat: (mostly) contains the RL terms:

    2.0D0   , start_temp:    Starting temperature.
    0.9995D0, temp_factor:   Annealing factor.
    0.01D0  , learning_rate: Learning rate.
    0.01D0  , alpha:         Decay rate for discounting purposes.
    1.0D0   , slope:         Slope of the sigmoid function.
    100     , save_interval: How often to save the weights.
    1       , rand_type:     0 = for testing.  1 = for training.
    1       , traffic_type:  0 = Down traffic only.  1 = Up and down.
    1       , inter_traffic: 0 = No inter-floor traffic.  1 = Yes.
    0       , reward_type:   0 = Omniscient rewards.  1 = Online rewards.
    0       , do_floor: (not used) 0 = Don't collect floor stats.  1 = Do
	      collect.
    0       , do_parking: (not used) 0 = Don't park.  1 = Do park.
    0       , do_sarsa:      0 = Q-learning.  1 = Sarsa.
    1.0D0 , lr_factor:     Learning rate factor.



OUTPUT: Besides the simulation graphics, there are three data files
   written to:
1) wait.stats - For each run, writes the run number and the total squared
   weight.

2) critic.stats - For each run, writes the run number and the average
   epsilon value.

3) weights - Weights of the neural network 

* You can get some additional output by turning on debug (changing the
  value to 1) in input.dat.




