Category Archives: Distributed Systems

Running the Ocaml Server – Demo Pt 1

 

I’ve got some code here and here in OCaml & Java, that I have been tasked with running, testing and fixing. I’ve already taken a look at the basic syntax but now its time to get to grips with some the libraries and related packages that Signposts makes use off.

THE SERVER IN OCAML

The first thing that took me by surprise was the number of files in the repo, considering that this is suppose to be a fairly straight forward project.

Oasis

The repo contains a file called “_oasis”, which on closer inspection includes a collection of metadata on the project. OASIS is a tools to integrate a configure, build and install system into an OCaml project. The _oasis files tells me that the entry point for the program is “server.ml” and the build dependences are the findlib packages lwt.syntax, lwt.unix, re, re.str.
The creater of the program will have ran “oasis setup”, which has generated setup.ml, _tags and myocamlbiuld.ml.

According to the instructions for OASIS you can configure, build and install the Ocaml program using:

  • ocaml setup.ml -configure

  • ocaml setup.ml -build

  • ocaml setup.ml -install

Re_str

The server.ml file makes use of Re_str and two of the biuld dependences are the findlib packages re and re.str. I have found the packages here on github. Re stands for Regular expression and ocaml-re is a regular expression library doe Ocaml (that is still under development). This project has also made use of OASIS and installs without a problem

Compiling the Code

I cd into the directory and run ocaml setup.ml -configure, ocaml setup.ml -build and ocaml setup.ml -install. This generates a new dirctory “_biuld” and a new link to an executable in _biuld

Running the Code

I run a link to the execuated in _biuld and it return the following error:
Fatal error: exception Unix.Unix_error(50, “bind”, “”)
This means that the socked that the code is try to bind to is already in use. I inspected the code, identified the ports that were being used and ran “sudo netstat -ap | grep ‘: ‘ “. This identified a process already using the port, the process was in fact an earlier attempt to run the project. I identified the process ID and killed the process, re-running “sudo netstat -ap | grep ‘: ‘ “ showed that the port was now free. I can now happily run the executable in the _build directory. As this is a client-server implementation, I now need to move my attention to the client code

Running the Android Client – Demo Pt 2

[ Sorry but this is part 2 in the series, I have completed part 1 but will not be able to upload the article until tomorrow ]

This is the 2nd part of a series on my taking a look at the code here, to test it and then improve it if required. In part 1, I ran the Ocaml server on my laptop. In this part, I run the android client. Tomorrow (Monday), in part 3 I am going to get the client and server communicating. On Tuesday, I will hopefully collect some data that I can then extract in some suitable format. On Wednesday, I hope to run Iperf between my client and server to collect data and extract in some suitable format. On Thursday, I hope to have a day of collecting data using both the GitHub code and Iperf. Finally on Friday, I hope to use a statistics package such as MatLab to compare the data collected and see if there is a statistically significant difference.

Since I have managed to mess up my eclipse/android install (see part 1), this gives me a perfect excuse to set up eclipse with ADT Plugin on my desktop. I’ve also got my hands on a different android phone, since the code that testing was designed for API 10. The phone info is as follows:


Model: HTC Hero Andriod Version: 2.3.7CPU: ARMv6- compatible processor #34Available memory: 190MBMod Version: CyanogenMod-7.2.0-RC1-UNOFFICAL
Build Number: GWK74

THE BIG PLAN

The stages in setting up the Android ADK, Eclipse and loading the code that I’m testing on the Android device (spec above)

  1. Download the Android SDK (Software Development Kit)
  2. The SDK only installs the SDK tools so to install other parts of the SDK, I’ve got to open up the Andriod SDK Manager, this is done by cd’ing into the tools/ directory and executing android SDK
  3. To develop an Andriod app, I need to download the lastest Android SDK Platform-tools and at least on Android platform. The platform that I want is the one associated with Android 2.3.7 and API 10, I’m going to get the related SDK Platform. Documentation and System image.
  4. Download and install the latest Eclipse Classic
  5. Add the source of the ADT Plugin to the Repository in Eclipse and download the Developer Tools in via Eclipse
  6. Specify the location of the Android SDK directory in the Preferences panel of Eclipse
  7. Create a AVD (Android Virtual Device) in Eclipse with target platform 2.3.7 and API 10
  8. Download the code that I’m testing from GitHub
  9. Import the code into Eclpise
  10. Test Code on the AVD
  11. Plug in Android phone and test code on the Andriod phone

AND THEN THE REALITY

1

The Android SDK is avaliable from the Android Developer Site, the package for Linux is called android-sdk_r20-linux.tgz.

The download comes as a .tgz file, this can be extracted using: tar -zxf android-sdk_r20-linux.tgz after using cd to move to the directory that you downloaded the file to.

I’ve found that its useful the learn as your going along, the purpose of the arguments that you are passing to programs, so that you can then easily adjust the arguments in future to suit your needs. Using man tar, explains that -z means filter archive through gzip, -x means extract the filesand -f means that your next going to give the file name.I would highly recommend that when following tutorials with Unix commands that you use man to look up what the executable and its arguments do.

2

As with the last time I installed Android SDK, to launch the Android ADK Manager, I need to enter ./android SDK instead of android SDK. This then appears to work fine, but I would still be interested as to why I didn’t work as expected.

[To clarify This command is entered after you use cd to move into android-sdk-linux/tools]

3

I selected the required packages (that I’ve listed in the plan above) and download them, simple as.

4

(Read this whole paragraph before beginning to download Eclipse) I download Eclipse Classic 4.2 for 32 bits as a .tar.gz file and extract the file (like before using tar -zxf eclipse-SDK-4.2-linux-gtk.tar.gz ). To install Eclipse, I cd into the new eclipse directory, to find no configure file, so my normal method of installing from .tar.gz files, of using ./configure && make && sudo make install will now not work. I little bit of research highlights that installing eclipse its not as straightforward as it might seem and the best approach for integration into Ubuntu is to get it from the software center, though I would prefer to continue to use the command line, I’m happy to leave it for now and return to this point at a later date

5

I start up eclipse, select my workspace and go into Help > Install New Software. I add the ADT Plugin at the URL https://dl-ssl.google.com/android/eclipse/. I then select the Developers Tools and install

6

With a slow connection,

you could be waiting sometime

I restarted Eclipse and configured the new ADT Plugin to link it to the Android SDK. On launching Eclipse, I’m presented with a Welcome to Android wizard (not used in the instruction at https://developer.android.com/sdk/installing/installing-adt.html)

7/8/9

Using the Eclipse GUI setting up a virtual device is straightforward. I navigate to the correct page on GitHub, download the code and extract as already done twice before and then I import the code as an existing Android project into Eclipse.

UNEXPECTED DIGRESSION

On loading the code, that I know should compile, eclipse highlights 2 errors and 13 warnings. As I’m just testing the code today before reviewing it next week. I will ignore the warming but deal with the errors as they prevent me from testing the code. The first error is from the line: import com.google.gson.Gson; I download the Gson project from googlecode and add it as an external libray to the project to overcome this. The second error is from the following line: public void onClick(View v). The error is that: The method onClick(View) must override a superclass method so this can be fixed by removing the above @Override annotation

10

Before I can run the code on the emulator, I create a new andriod project and add the code so far. I then use Run As > Android Application to test the code and it works

11

I put the android phone into USB debug mode, plug it in and run as. Again it works 🙂

“The best part about being a computer scientist is the feeling of satisfaction when something finally works”

I’m back from brussels

From this …

I arrived in the back in Cambridge yesterday afternoon after the IBM EMEA Best Student Recognition Event 2012 in Brussels, Belgium. The theme of this years 3 day event was “Data Analytics”. The highlight for me on Wednesday 4th July was the a semenar by Prof. Bart Goethals, a lecturer at University Antwerp on data mining and machine learning in data analytics. For Thursday 5th July my highlights were a talk by Corinna Schulze from the IBM Governmental Program on privacy and a talk by Robert-Jan Sips focusing on how data analytics is being applied to water management in the Netherlands.  And for the final day was highlights was a talk by Guy Lefever on the application of Watson to future medical care. Overall I really enjoyed the event and I would throughly recommend attending if you get the opportunity to go.

It looks as if I’m going to have some new work on Signposts next week but I don’t know as yet what this will be so over the weekend I intend to review some of the new material that I’ve learned over the my first few weeks on the Undergraduate Research Opportunities Project (UROP) here at the William Gates Building, Cambridge. My initial project brief was:

Signposts – the virtual personal internet 

Users with multiple networked devices are having a hard time linking them up when said devices reside at more than one location. Simple example is trying to synch your android phone while on the bus to the laptop in your office and the Mac Mini at home. NATs and Firewalls and other obstacles to symmetric communication abound, and obstruct. Point solutions (Dropbox, iCloud) do not deal with many cases (different OS) and only work if you pay (with money or eyeball time on adverts). In any case, they are not acceptable to privacy conscious users.

Instead, we are building a system called signposts. This is the smallest conceivable virtual machine that sits in the public internet, and looks after your personal namespace of devices. Registration and lookup of names is “effectful”, in other words, has side effects. The side effect is to use a set of possible tactics to rendezvous between the devices (various techniques for NAT traversal, for example) including possible multiple simultaneous path use, and also allowing for constraints, such as battery life, and price of use of connections. The system sits on top of DNSSEC, but more interestingly, as a piece of Computer Science, makes use of the new Cloud OS software being developed in CL, called Mirage (see http://www.openmirage.org/ and https://github.com/avsm/signpostd

This project is to help with the documentation and testing “in the wild” of signposts. We expect to uncover a lot of problems with different oddities (see http://conferences.sigcomm.org/imc/2011/docs/p181.pdf for examples) that appear, and we need to have good stories to account for what we are doing to the public and to security people. OCaml programming is pretty essential, as well as at least basic understanding of development tools (including git etc) “

Over the last few weeks, I had used a wide range of tools that are new or that I had not previously used to this extent including (in no particular order):

  • Unix command line – although this is something that i typically use, I have definitely used it more and for a wider range of applications that ever before
  • Git & GitHub – for my 2nd year group project PROTON, we took the (controversial) decision to use Mercurial and BitBucket instead of Git and GitHub like most other group so it been interesting to see how they compare
  • Tor – As alot of my work over the last few weeks has been looking at testing the performance of tools for online anonymity, I’ve made extensively use of the onion router
  • Orbot – Tor for android
  • Tails – An OS build around tor
  • Iperf – As just explained as I’ve been working with testing network performance Iperf has been an invaluable tool (though I have recently discovered Torperf)
  • Amazon Elastic Compute Cloud – I’ve been using this a server when testing network performance, its really use as it provides as global domain name and IP address and avoid pathological cases that could be created by having other cleint and server on the same network
  • SHH – for remote access to the other computers I am testing with locally and the virtual machine on Amazon EC2
  • dig-host – command line tools for performing DNS queries, just generally useful
  • CyanogenMod – the OS on my android phone which allows easy using of proxies and tethering
  • Privoxy – heavy customizable non-caching web proxy
  • Hamachi – used with privoxy so set up encrypted VPN access
  • OpenVPN – easy set up of VPNs
  • Texmaker – a nice LaTeX editor, that I would throughly recommend which I have been using to keep the notes on my work over the last few weeks
  • Mozilla Firefox & addons – previously a Chrome user myself, I’ve opted to use Firefox since the research project begin for a change and during this time I have experimented with a range of addons for privacy including: HTTPS Everywhere, NoScript,  AdBlock Plus, DoNotTrackPlus, Collusion …
  • Eclipse & Android Development tools – getting to grips with the basics of Android development in preparation for the Google European Android Camp
  • Telnet – for commend line control of Tor
  • Wireshark – very useful for learning about computer networking, allows you to watch the packets on the network
  • TraceRoute – brillient tools which used TTL to should the route that a packet takes to its destination
  • Ocaml – the main project that I’m working with over the summer makes extensive use of Ocaml

I plan to use this list to visit some of the material that I’ve learned over the last few weeks in preparation for new work next week.