Category Archives: Programming Languages

Profiling OCaml – Getting Started Guide

Leave a reply

“Perf” is a common command line linux tool used for code profiling, (perf wiki). A alpha version of a OCaml native code compiler that output code, that can be analysis by perf is now avalaible in OPAM

Installation

Installing the perf compatible OCaml compiler is straight forward with OPAM, though quite time-consuming due to the need to re-install many packages

$ opam remote add perf git://github.com/mshinwell/opam-repo-dev
$ opam switch 4.01-perf-annotate
$ eval `opam config env`
$ opam install

Installing perf was also straight forward, in fact I already had it via the linux-tools package in apt-get for Ubuntu.

sudo apt-get install linux-tools

Usage
Compiling with the new perf-compatable Ocaml compiler was beautifully simple, running make within an existing project working first time without any further changes necessary.

Basics
Basic reporting is collected and viewed using:

sudo perf record ./myprogram.native -o myreport.data
sudo perf report -i myreport.data
sudo perf script -i myreport.data

Similarly basic stats can be collected using:

sudo perf stat ./myprogram.native -o myreport.data
sudo cat myreport.data

When finished you can switch back to your normal compiler version, i.e.

$ opam switch 4.00.1
$ eval `opam config env`

Learning Async

Leave a reply

If your taking your first steps in Janestreet’s Async library, here’s some sources of help & support:

My top 5 sources of Async Documentation

  1. Official Janestreet Docs for Async/Core: https://ocaml.janestreet.com/ocaml-core/latest/doc/ These are generated from the .mli’s in the Async source and you will quickly find places where ocamldoc has failed to cope with Core/Async complex module structure. In this case, I recommend the source code: https://github.com/janestreet/async
  2. Dummy’s guide to Async, a useful intro to bind, upon and map (the 3 fundemental functions): http://janestreet.github.io/guide-async.html
  3. Real World OCaml chapter on Async, covers Deferred basics, Ivars, Reader/Writer, TCP server, running the scheduler, Command, Pipes, Monitors, Delays, Choose and System Threads: https://realworldocaml.org/beta3/en/html/concurrent-programming-with-async.html
  4. Yaron Minsky (one of co-authors of RWO) provides a nice little example of Async’s RPC libraries: https://bitbucket.org/yminsky/core-hello-world
  5. My favourite project for example Async and Lwt code is cohttp: https://github.com/mirage/ocaml-cohttp, otherwise there’s OPAM’s list of Async dependent project, useful for finding more example code: http://opam.ocamlpro.com/pkg/async.109.38.00.html

Community

Google group for Janestreet’s Core & Async, the JS team are really helpful 🙂 : https://groups.google.com/forum/#!forum/ocaml-core

Async issue tracker on Github is quiet but still a good point to start a conversation: https://github.com/janestreet/async/issues?state=open

Installation

I recommend the installation chapter of Real World Ocaml: https://realworldocaml.org/beta3/en/html/installation.html and install instructions for OPAM: http://opam.ocamlpro.com/doc/Quick_Install.html

Janestreet provide some installation hints, though this is covered for comprehensively in RWO (linked above): http://janestreet.github.io/installation.html

Very light background reading: https://ocaml.janestreet.com/?q=node/100

 

 

System programming in OCaml – Part 2

Leave a reply

Now we are going to take a look at manipulating files in OCaml. We will be focusing on the Unix file system.

In Unix the term “file” refers to a greater range of objects that you might expect. This includes:

  • standard files like .txt, .html, .doc or .ml
  • directories (or folders if your used to windows) like /photos or /music
  • symbolic links (or shortcuts if your used to windows)
  • special files to access devices i.e. files in /dev
  • named pipe ( this is a named version of using | in the terminal )
  • sockets

All files are represented by an i-node, a data structure which holds the meta-data on the file. Directories are files that map filenames to i-nodes. The directories form a tree structure, where all files are directly or indirectly children of the root directory called ‘/’.

A absolute path name of the file is one that begins at the root. A relative path name is one that begins in the current directory. Every directory has ‘.’ and ‘..’ which are symbolic links to the current and parent directories.

OCaml makes use of the Filename module to help handle paths in a portable manner

Ocaml 2 HTML

Leave a reply

As I’m often adding OCaml code to blog posts I really wanted to find an automatic way to add syntax highlighting. The first solution that I have found is caml2html but I’m still looking for something better … suggestions welcome

System programming in OCaml – Part 1

Leave a reply

This series of articles will follow my journey through “Unix system programming in OCaml”, available as a pdf here. After this series, I hope to move onto a series on Mirage system programming in OCaml and working with Signposts (a framework for managing end-to-end connectivity between devices using DNS infrastructure for signalling)

Introducing the Sys and Unix

The modules that give access to the system in OCaml are:

  • Sys – functions common to all OS’s that run OCaml
  • Unix – functions specific to unix

To make use of these modules in some OCaml code, we can add open Sys or/and open Unix to the top of the OCaml program.

NOTE that Unix and Sys can overwrite some of the functions in the Pervasive module, so to access a function from the Pervasive module you need to add Pervasives. in front of the function call.

When using ocamlc to compile OCaml programs that made use of  Unix, you need to pass unix.cma to the compiler e.g.

$ ocamlc unix.cma hello.ml -o hello

The same is not true of sys.cma

PASSING INFORMATION TO YOUR PROGRAM

Now we are going to consider 2 different ways of passing information from your terminal to your ocaml program

1) Passing the information as arguments to the executable. For this we are going to use Sys.argv which will return a string array of arguments passed to the program. The first index of the argument array that we use here is 1, this is because the 0th element in the array is the command used to call the program. Consider the following simple example:

open Sys;;
print_string argv.(1)
Now provided that at least one argument is passed to the program, this argument will be printed. We can now extend this program, to print the number of arguments and then exactly what the user will have just entered
open Sys;;

let args = Array.length argv;;
print_int args ;;
print_newline ();;
for i=0 to args-1 do
print_string argv.(i);
print_char ‘ ‘;
done;

2a ) Passing information using the environment variables, this can be fetched from within the OCaml programing using Sys.enviroment, this will return as array of environment variables

open Unix;;

let env = environment();;
for i=0 to (Array.length env) -1  do
print_string env.(i);
print_newline);
done;

Don’t forget then when you compile this, you need to add unix.cma to the command as we are making use of the Unix library

2b ) We can also lookup the values of environment variables using the Sys.getenv (or Unix.getenv) functions that takes a variable name and returns the value associated with it

open Sys;;

let var = argv.(1);;
print_string (getenv var)

Exceptions

Unless otherwise stated in the documentation, all function in the Unix module raise the exception Unix_error.

Exception Unix_error of error * string * string

The first argument is the error code, the second argument is the name of the system call that raised the error and third argument identifies the object on which the error was called.

If an exception reaches the top-level then the program will halt. The Unix module provides a function for clearly describing exceptions. We can make 2a use this function as follows:

open Unix;;

let prog () = 
  let env = environment();;
for i=0 to (Array.length env) -1  do
print_string env.(i);
print_newline);
done;
 done;

 handle_unix_error prog ();;

Getting to grips with OPAM

Leave a reply

What is OPAM ?

OPAM is a OCaml Package Manager. Its basic use is similar to apt-get on linux, both package managers help to automate the process of installing, removing and updating software by tracking the complex web of dependencies that exist between bodies of code.
Why use OPAM ?

OPAM gives the user more control of the choice of specific libraries and compiler versions used to run OCaml code.

How to do I run OPAM ?

I am only going to run OPAM and not install it (long story). To do this, I executed the following from my root directory

wget http://opam.ocamlpro.com/build/opam64

chmod +x opam64

./opam64 init

How can install a package using OPAM ? What is the advantage over using OPAM intend of using the traditional methods ?

You can view the list of available packages using ./opam64 list and you can search using ./opam64 search .

On of the libraries used by Signposts is lwt. Lwt is a very useful library for anyone who want to do multi-threading in Ocaml. So I try the following query:

opam64 search lwt

Available packages for system:

cohttp   --  HTTP library for Lwt, Async and Mirage

lwt   --  A cooperative threads library for OCaml

lwt-zmq   --  Lwt-friendly wrapping around ZeroMQ sockets

release   --  Release is a multi-process Lwt-enabled daemon framework for OCaml.

The second of these results is the package that I want to install so

How do I install a package from a Ocaml project on Git ?

Firstly find the package that you want to install, for me this is git://github.com/mirage/opam-repo-dev

opam remote -add dev git://github.com/mirage/opam-repo-dev
Now in future, when I issue opam upgrade, the latest version of the git repo will be downloaded and any depandencies recompiled

Running the Ocaml Server – Demo Pt 1

Leave a reply

 

I’ve got some code here and here in OCaml & Java, that I have been tasked with running, testing and fixing. I’ve already taken a look at the basic syntax but now its time to get to grips with some the libraries and related packages that Signposts makes use off.

THE SERVER IN OCAML

The first thing that took me by surprise was the number of files in the repo, considering that this is suppose to be a fairly straight forward project.

Oasis

The repo contains a file called “_oasis”, which on closer inspection includes a collection of metadata on the project. OASIS is a tools to integrate a configure, build and install system into an OCaml project. The _oasis files tells me that the entry point for the program is “server.ml” and the build dependences are the findlib packages lwt.syntax, lwt.unix, re, re.str.
The creater of the program will have ran “oasis setup”, which has generated setup.ml, _tags and myocamlbiuld.ml.

According to the instructions for OASIS you can configure, build and install the Ocaml program using:

  • ocaml setup.ml -configure

  • ocaml setup.ml -build

  • ocaml setup.ml -install

Re_str

The server.ml file makes use of Re_str and two of the biuld dependences are the findlib packages re and re.str. I have found the packages here on github. Re stands for Regular expression and ocaml-re is a regular expression library doe Ocaml (that is still under development). This project has also made use of OASIS and installs without a problem

Compiling the Code

I cd into the directory and run ocaml setup.ml -configure, ocaml setup.ml -build and ocaml setup.ml -install. This generates a new dirctory “_biuld” and a new link to an executable in _biuld

Running the Code

I run a link to the execuated in _biuld and it return the following error:
Fatal error: exception Unix.Unix_error(50, “bind”, “”)
This means that the socked that the code is try to bind to is already in use. I inspected the code, identified the ports that were being used and ran “sudo netstat -ap | grep ‘: ‘ “. This identified a process already using the port, the process was in fact an earlier attempt to run the project. I identified the process ID and killed the process, re-running “sudo netstat -ap | grep ‘: ‘ “ showed that the port was now free. I can now happily run the executable in the _build directory. As this is a client-server implementation, I now need to move my attention to the client code

Getting Started with Python

Leave a reply

Why have I chosen to learn Python ?

  • Its a language sometimes used by my summer research project
  • It works particularly better than other language on the Raspberry Pi
  • I’ve covered the basics of Python before but never gone further than simple syntax
  • Its open source
  • It can be used as a scripting language, which is an area that I really want to improve
  • It supports object-oriented programming so I feel that with my previous experience of Java and C++, I will have covered the most popular OOP languages

Python 2.7 or 3 ?

The first decision that I needed to make before learning Python was whether to learn Python 2.7 or 3, since Python 3 fixes some of the “non-optional” design decision found in 2.7 and it of course more update I decided to learn Python 3. The potential problems with this is the lack of backwards compatibility with 2.7 and lack of teaching materials.

Python in my degree ?

The lecture course which is most likely to have included Python is Concepts of Programming Language which I took in Easter term this year. The course makes a few references to the language but does not cover the language to the degree that languages such as Fortran, Lisp, Algol, Pascal, Simula, Smalltalk, SML and Scala were covered.

What programming paradigms does Python support ?

Python advertises itself as a multi-paradigm programming language and from what I can see the range of supported paradigms supports this claim. This makes Python a multi-purpose language along with languages such as Java, C, C++ and SML compared to special-purpose languages such as SQL and LaTeX. The paradigm which Python support include:

  • imperative programming – describing computation by prescribing how to move the program from one state to another, imperative languages included Fortran, C and assembly
  • object-oriented programming – it supports dynamic lookup, abstraction, subtyping and inheritance, OOP language include Java, C++, Simula, Smalltalk and Scala
  • Functional programming – treats computation as the evaluation of mathematical function and avoids state and mutable data. Examples include Lisp, Ocaml, Haskell and Scala
  • Reflection programming – allowing a program to examine and modify its structure at runtime. Like Java, ML and Haskell
  • Structured programming – subroutines, blocks, for and while making code more readable compared to simple tests and goto’s. Examples include most modern languages except assembly code

 How does Python handle types?

Type checking is the process of attempting to prevent type errors by ensuring that the operations in a programm are applied properly. There are two common types of type checking:

  • Dynamic (run-time) type checking – the compiler generates code such that whenever an operation is performed when the program is ran, the code checks to make sure that the operands have the correct types. Examples of dynamically type checked languages are MATLAB, Prolog and Ruby
  • Static (compile-time) type checking – the compilers checks that program text for potential type errors as the program is compiled. Examples of statically type checked languages are  Fortran, Haskell, Java and Ocaml
Python uses Duck typing

Python is dynamically type checked, in fact it uses duck typing. Duck typing means that valid semantics of a object are dictated by the current state of methods instead of its inheritance from a class/interface like in Java. Why call it duck typing then ? the name refers to the duck test:

“When I see a bird that walks like a duck and swims like a duck and quacks like a duck, I call that bird a duck”

This allows use of EAFP or “It’s easier to Ask Forgiveness than Permission”. attributed to Grace Hopper.

A type system is said to be strongly typed when it specifies restrictions on how operations involving values of different data types can be intermixed. If this is not true, then we describe the language as weak. Python is strongly typed.

The built in types are as follows:

  • bool – truth value e.g. True or False
  • int – int of unlimited magnitude
  • list – mutable sequence (supports mixed types) e.g. [4, 4.0, “Four”]
  • tuple – immutable sequence (supports mixed types) e.g. ( 4, 4.0, “Four”)
  • complex – complex number e.g. 3+5j
  • float – floating point number e.g. 3.14
  • dict – group of key and value pairs {“key1”:1.0, “key2”:2.0}
  • set – unordered, no-duplicate collection of values
  • bytes – immutable sequence of bytes
  • bytearray – mutable sequence of bytes
  • str – sequence of Unicode characters

Mutable means that an objects state can be modified after its created, immutable means the opposite.

Python uses the off-side rule

Syntax & Semantics of Python ?

Unlike almost all other languages that I have studied Python uses whitespace to delimit block instead of curly braces (like Java), this feature is termed the off-side rule.

You define a function or method using the def keyword. The typically statements that you would except also apply: if, for, while, try, except and finally.

Python uses the words and, or, not for boolean expressions. Integer division (using //) is defined to round towards minus infinity and floating point division is do using /. Compare by value is achieved using == and compare by reference is achieved using is.

The syntax and semantics of Python highlight the main purpose of Python as a language for teaching programming, hence the focus on readability. This is why Python is the language being taught to school children using the Raspberry Pi

Installing the Interpreter

I install the packages for Python 3 from the Ubuntu Repositories using  sudo apt-get install python3-minimal, and then run the interpreter by entering Python3 into the terminal

Once I get going with Python programming I hope to return to this type of language analysis and review how to design decisions made in the development of Python 3 reflect the use of the language in an introduction to programming. I hope to put up a part 2 of Getting Started with Python, later this week.

As ever, feel free to comment and highlight my mistakes even the spelling/grammar ones.