CSCI 4534 - Operating Systems

Assignment 1 Requirements

Compute server

A compute server is a program that typically runs on a powerful supercomputer and does number crunching, i.e. runs jobs that require a lot of CPU power. Jobs are submitted to the server, and usually each job comprises one algorithm as well as a series of numerical inputs; the server applies that single algorithm to each input in turn, and spits out a series of numerical outputs.

Your task is to build a compute server for NASA on your desktop machine, as well as a client that talks to this server. Since you've just been hired at NASA, you are just a junior engineer, and therefore your mentor provides you guidance as you build this system. This guidance is in the form of an overall framework, i.e. a set of Java interfaces and some classes, as well as some hints and suggestions. Your task is to write Java classes that implement those interfaces.

First off, your mentor wants to ensure that you remember basic Java. To this end, you are given the following:

The interface csci4534.calculator.Calculator.
Two simple implementations of this interface: csci4534.calculator.SeriesCalculator and csci4534.calculator.FactorialCalculator.
A simple command-line driver csci4534.calculator.MainCalculator which applies the calculator to an input.
The batch file make.bat to build your code.
The batch file jdoc.bat to produce HTML documentation from your code's javadoc comments.
The batch file compute.bat to run MainCalculator.

Build the provided code using make.bat, and run

compute.bat FactorialCalculator 15

until you get the correct response of

Calculator: csci4534.calculator.FactorialCalculator
Result on input 15 is 1307674368000

[3 points] Study the provided implementations of Calculator. Write a new implementation BitCalculator which, given input n, it computes the number of bits set to 1 within the binary representations of all the integers from 0 through the n. For example,

n	Result	Explanation
0	0	No 1's
1	1	1 is 0001 in binary
2	2	2 is 0010 in binary so we get one more set bit than in the previous row
3	4	3 is 0011 in binary which adds two more set bits

Run the batch file jdoc.bat to create the HTML documentation of your code. Make sure your code is well documented in the same simple, succinct style that the provided code is documented. As the syllabus states, points will be deducted for poorly commented code you write for this or any other programming task in this course.

Now that you have refreshed your Java skills, it is time to start working on the server itself. Your mentor again provides you some starting material:

The interfaces csci4534.server.Server and csci4534.client.Client.
A simple command-line driver csci4534.server.MainServer which executes the server.
A simple command-line driver csci4534.client.MainClient which executes the client.
The batch file runserver.bat to start the server.
The batch file runclient.bat to run the client.

Your next task is to implement the provided interfaces. Your end goal is to be able to run

runserver.bat 5135

in one terminal window (which blocks and only print status messages), and then run

runclient.bat localhost:5135 BitCalculator 3

in another window. The client connects to the server, passes on the desired algorithm (BitCalculator) along with the numeric input (3), receives a response like

Result on input 3 is 4

and finally quits. (5135 is the port number and it's arbitrary: most port numbers will work --- unless in use by other applications --- as long as you use the same port for both client and server.) The server continues to run, and services other clients that connect to it. To achieve this end goal, start by reviewing Java's serialization mechanism, and specifically the JDK javadoc on Serializable, ObjectInputStream and ObjectOutputStream in the java.io package. Pay particular attention to the following facts:

The constructor of ObjectInputStream may block if there's no data in the underlying stream.
ObjectOutputStream does not automatically flush itself when new data is written to it.

Then, review the comments in the provided Client and Server interfaces. Focus on the basic communication protocol, and ignore (for now) the use of csci4534.server.EndCalculator, multi-threading, and caching.

[15 points] Implement a single-threaded socket-based client in csci4534.client.SocketClient, and a corresponding single-threaded socket-based server in csci4534.server.SocketServer.

The first improvement to this basic implementation is a simple mechanism to enable the client to stop the server. Without this improvement, the server runs forever unless you kill its process; this is not only clumsy, but it can lead to system instability as an assassinated process does not have a chance to clean up after itself. So, we introduce the provided EndCalculator. If a client sends this dummy calculator instead of a real algorithm, the server gracefully exits.

[10 points] Add support for EndCalculator in SocketClient and SocketServer.

You can test your latest feature by running

stopserver.bat localhost:5135

instead of a regular client. Make sure that no stray exceptions get thrown by either client or server.

Your NASA mentor points out that scientists are unorganized people. Often, two scientists in the same team want to perform the same computation, but they don't coordinate. So they instead ask their team leader to submit a client job like this one:

runclient.bat localhost:5135 SeriesCalculator 1000000000 1 2 3 1000000000

The NASA supercomputer does the same long job twice only to produce the same answer! Naturally, the next improvement to your implementation is to add caching. A simple way to do this is by implementing csci4534.server.CachingCalculator, which is really a caching proxy: it doesn't directly do any specific computation. Instead, it delegates the computation to another calculator iff the input is a new one; otherwise, it simply returns the result that had been computed earlier, when the same input was first received. Unlike all other calculators, CachingCalculator is not a pure functional object: it has instance variables, where (among other things) it keeps track of past inputs and results.

[10 points] Implement CachingCalculator. Note that CachingCalculator should be a proxy around any other calculator instance of any class. Hint: you might find java.util.Hashtable very helpful.

[5 points] Integrate CachingCalculator into SocketServer so that computations of a single client are cached throughout the lifetime of its connection to the server.

With your changes, the first input may take a while to produce an output, but the last input produces a result immediately.

One fine day, as you are playing Space Invaders on your NASA office computer, your mentor runs in screaming that China and Taiwan are about to go to war unless you save the world! Naturally you panic, and immediately hide the Space Invaders window and bring up your command shell. Then you realize that a war is more important than workplace image, so you regain your cool and ask your mentor what you can do to avert disaster, and here is his response: "The Chinese and Taiwanese scientist teams both want to use the supercomputer at the same time, but your compute server can only run one job at a time. Neither team is willing to back down and let the other team go first. Can you fix it?" Naturally, you exclaim "Yes, I can fix it thanks to CSCI 4534!" and get to work.

So the final improvement to your implementation is adding multi-threading. In other words, every time a client connects to the server, the server should spawn a new thread to handle processing of all numerical inputs of this client, while the main thread of the server goes back to waiting for other clients. There is a small catch though: the JVM on the supercomputer has a bug whereby the Thread class is declared final. This means that you can use it in all other ways except extending it.

[15 points] Add multi-threading to SocketServer.

Finally, you rush into your mentor's office (where you catch him playing Space Invaders too), and demonstrate your solution by starting the server in one terminal window, and on two other terminal windows you run one client each:

runclient.bat localhost:5135 SeriesCalculator 1000000000

and

runclient.bat localhost:5135 BitCalculator 50000000

Both jobs complete after around the same time interval. You avert war, and your mentor and you celebrate by playing a 2-player game of Space Invaders.

[1 point] How much time did you spend on this section?

Compute server analysis

[2 points] Try running your client on a separate machine than the server. Should it work? Why, or why not? Show the command-line you used on each machine.

[2 points] Try running your client against a server built by another student. Should they work together? Why, or why not?

[5 points] What would happen if NASA exposed your server to the Internet, allowing thousands of people to submit their compute jobs at the same time? What thread organizing technique can you use to address this problem?

[5 points] What would happen if two different teams of scientists (i.e. two separate clients) submitted a request for the same algorithm and same numeric input? How would you fix this?

Job scheduling

Simulation is a very useful tool for evaluating CPU scheduling algorithms. Simulators come in many different flavors. For example, VMWare simulates the hardware of a virtual machine; since the CPU is simulated, VMWare can easily keep track of its utilization (though it would have a hard time mapping it to OS-level processes since VMWare doesn't have a clue what OS is managing the CPU). Alternatively, a simulator can be as simple as the one below.

You are given a simple simulator of CPU scheduling in csci4534.scheduler.MainSimulator. This simulator keeps track of a set of (simulated) processes running on a system. As (simulated) time passes, processes get scheduled to run on the single CPU. As different processes move into and out of the CPU, the simulator tracks their waiting time, and their remaining burst time. When all processes are done, the simulator reports the average waiting time of all processes and exits.

Which process has control of the CPU (i.e. CPU scheduling) is determined by a user-supplied instance of the csci4534.scheduler.Scheduler interface. For example,

schedule.bat FCFSScheduler 0:0:24 0:0:3 0:0:3

simulates the CPU behaviour under the setup discussed in the textbook section 6.3.1. There is one triplet of the form p:t_a:t_b for each process, where

p is the process priority; some schedulers ignore it. Use 0 for those examples in the book which do not list an explicit priority.
t_a is the process arrival time. Use 0 for those examples in the book which do not list an explicit arrival time.
t_b is the next burst of the process; for simplicity, you can simply think of this as the lifetime of the process, i.e. the process is completely ignored after it has spent a total of t_b time on the CPU.

For the above input, here is the output you get:

Scheduler: csci4534.scheduler.FCFSScheduler
Loaded (P1,0,0,24,0)
Loaded (P2,0,0,3,0)
Loaded (P3,0,0,3,0)
@0 started (P1,0,0,24,0)
@24 stopped (P1,0,0,0,0)
@24 finished (P1,0,0,0,0)
@24 started (P2,0,0,3,24)
@27 stopped (P2,0,0,0,24)
@27 finished (P2,0,0,0,24)
@27 started (P3,0,0,3,27)
@30 stopped (P3,0,0,0,27)
@30 finished (P3,0,0,0,27)
Average waiting time: 17.0

You are given the implementation of csci4534.scheduler.FCFSScheduler. Study it, along with the rest of the provided code, and then carry out the following tasks:

[10 points] Implement csci4534.scheduler.SJFScheduler.

[10 points] Implement csci4534.scheduler.PriorityScheduler.

[15 points] Implement csci4534.scheduler.RRScheduler with a time quantum of 4 units of simulated time. Hint: store in an instance variable the process you will return next time the simulator calls your implementation of choose().

[1 point] How much time did you spend on this section?

[1 point] Summarize in a concise list the key concepts and/or technologies that this assignment helped you digest. You don't have to explain the listed items; just name them.