LTSP-MOSIX

This page describe how to transform a LTSP network which consist in a server and several Xterminals into a Mosix cluster.

William Daniau : wdaniau@lpmo.edu

Current version

ltsp_mosix_core : 1.0beta4 with ltsp 3.0.5
Mosix kernels : 2.4.18 + Mosix 1.7.0
OpenMosix kernels : 2.4.22 + openMosix-2

Note : Mosix 1.8.0 and Mosix 1.9.0 have a bug that prevent ltsp_mosix kernels to boot. This appears if DFSA is activated (which is the case in ltsp_mosix), it hangs during pivot_root. This is the reason why the current version of ltsp_mosix is not the latest Mosix version.

Mailing-list

This mailing list is intended for general discussion and announcements.
subscribe at http://www.lpmo.edu/mailman/listinfo/ltsp-mosix

IMPORTANT NOTE FOR RELEASES PRIOR TO 2.4.22-openMosix-2

openMosix users must not use the rpm from openMosix site. The reason is that openMosix kernel options must be the same on every node, or you may experience crashes. Here are the ones of the ltsp-openmosix packages :

#
# openMosix
#
CONFIG_MOSIX=y
# CONFIG_MOSIX_TOPOLOGY is not set
CONFIG_MOSIX_SECUREPORTS=y
CONFIG_MOSIX_DISCLOSURE=1
CONFIG_MOSIX_FS=y
CONFIG_MOSIX_DFSA=y
# CONFIG_MOSIX_PIPE_EXCEPTIONS is not set
# CONFIG_openMosix_NO_OOM is not set
# CONFIG_MOSIX_LOADLIMIT is not set

you must compile a kernel with these options, in order to use the kernels from this site.

News

2003 December
- New kernels 2.4.22-openMosix-2 builds with same options as the rpms from openMosix site so they can be used with the rpms
2003 August
- New kernels 2.4.21-openMosix-1 and user tools 0.3.4
2003 April :
- OpenMosix port at last! (MosixScan has not been ported yet but may be soon.)
2002 September :
- At last some new packages! ltsp-mosix is now in version 1.0beta4 build upon ltsp_core_3.0.5, kernels are 2.4.18 with Mosix 1.7.0
- All old stuff has been removed to clarify the page.
- remote program introduced in 1.0beta2 has been removed, as I will now use Raphaël Armati's MosixScan client/server program.
- You can now subscribe to the ltsp-mosix@lpmo.edu mailing-list if you want to be informed on new versions, or discuss configuration problems etc...
2002 August : Finally some Mosix 1.7.0 stuff added here, it is not TESTED. I've got some problems with it on a dual Athlon Tyan motherboard. Please give me some feedback if you uses it.
2002 May : new kernels for Pentium4 and Athlon
2002 March : new beta release 1.0beta3, with kernel 2.4.17 and Mosix 1.5.7
2002 February : new beta release 1.0beta2
2002 January 18 : Here it is! ltsp-mosix-1.0beta1 for ltsp 3.0
2002 January 15 :just finish to port ltsp-mosix to ltsp3.0, not available for download now, cause I have to put it in a package, but it will be available very soon, keep an eye on that page ;-)
2002 January 4 : kernels for 3com 3c590/3c900 series
2001 December 17 : New version 0.3 with Mosix 1.5.2,kernel 2.4.13,mfs and dfsa
2001 September 4 : v 0.2 . Minor fixes
2001 July : First launch v 0.1

Changes

1.0beta4

Nothing really new, just rewrite scripts upon ltsp_core_3.05.
some news although in the kernels, I have found that some kernels (even the original ltsp_kernel) did not boot on machines with more than 1GB of Ram. This is solved in the new ltsp-mosix kernels by relocating the initrd.
Some problems with 2.4.19 and Mosix 1.8.0 as it hang during pivot_root, this is why I currently use 2.4.18 and Mosix 1.7.0

1.0beta3

For easier upgrading of Mosix in ltsp-mosix, it is now splitted into 4 files :
- ltsp_mosix_core : core package, mosix version independant
- ltsp_mosix_user_mosixversion : user utilities from Mosix version mosixversion.
- ltsp_mosix_kernel_Pentium_mosixversion : Pentium kernel with Mosix version mosixversion.
- ltsp_mosix_kernel_Pentium_SMP_mosixversion : Pentium kernel with SMP support and Mosix version mosixversion.
Added epic100 driver

1.0beta2

The kernel in beta1 was Pentium classic with SMP support. Some people have had some trouble with this, as on some uniprocessor configuration it will not boot. This beta2 release, is divided in a core package and some kernel packages (2 for the moment), with and without SMP.
The mechanism for starting scripts in rc.d has been modified, scripts are launched by init instead of rc.local as I experienced some problems with the original mechanism.
There is a draft quality remote control daemon called "remote" that you can activate using the rc.d scripts feature by adding
RCFILE_01 = start_remote
in lts.conf. This can be used to switch on/off mosix on a node as follow :
telnet node-ip 5998
then type "start" or "stop".
All contributions for something better will be welcome!

Downloads

To install ltsp on the server Not all the packages are here, so you are encouraged to go to ltsp home page.
To install Mosix on the server You are also encouraged to go to Mosix home page, but you have to use same versions of kernel and Mosix as below. You'll have to compile it with options MFS and DFSA.
- linux-2.4.18.tar.bz2. This came from kernel.org
- MOSIX-1.7.0.tar.gz
- MOSKRN-1.7.0.tar.gz
To install OpenMosix on the server : please visit openmosix home page to get full instructions, configure the kernel with MFS (DFSA is not visible in the make xconfig interface but will be automatically checked on)

When installing Mosix on the server, as stated in Mosix instructions, it is a good idea to modify your /etc/inittab to lock processes on the server, for example :

change
/etc/init.d/rc 5
into
/bin/mosrun -h /etc/init.d/rc 5

this way all critical processes will be locked to the server. This is needed because with Mosix there is no failure protection, that means that if a process is currently migrated to work station 1 and somebody suddenly poweroff that workstation then the process is dead.
When you want a process to be able to migrate, launch it by prefixing it with mosrun -l

Common LTSP-MOSIX package for Mosix and openMosix

Do NOT mix Mosix and openMosix stuff

ltsp_mosix_core-1.0beta4.tar.gz: add Mosix/openMosix to ltsp

LTSP-MOSIX packages
- ltsp_mosix_user_1.7.0.tar.gz User utilities (required).
- ltsp_mosix_kernel_Pentium_1.7.0.tar.gz: kernel and modules for Pentium classic without SMP.
- ltsp_mosix_kernel_Pentium_SMP_1.7.0.tar.gz: kernel and modules for Pentium classic with SMP.
- ltsp_mosix_kernel_Pentium3_1.7.0.tar.gz: kernel and modules for Pentium III without SMP.
- ltsp_mosix_kernel_Pentium3_SMP_1.7.0.tar.gz: kernel and modules for Pentium III with SMP.
- ltsp_mosix_kernel_Pentium4_1.7.0.tar.gz: kernel and modules for Pentium4 without SMP.
- ltsp_mosix_kernel_Pentium4_SMP_1.7.0.tar.gz: kernel and modules for Pentium4 with SMP.
- ltsp_mosix_kernel_Athlon_1.7.0.tar.gz: kernel and modules for Athlon without SMP.
- ltsp_mosix_kernel_Athlon_SMP_1.7.0.tar.gz: kernel and modules for Athlon with SMP.
LTSP-openMosix packages : If you prefer to use openmosix, you must install the same ltsp_mosix_core package as for mosix, the ltsp_openmosix_user package and a kernel package. Everything is configured the same way as for mosix except the configuration file which must be hpc.map instead of mosix.map
- uninstall_mosix_user.sh: if moving from a mosix installation, use this script to uninstall ltsp_mosix_user before installing ltsp_openmosix_userland
- ltsp_openmosix_userland_0.3.4.tar.gz (required)
- 2.4.20-openMosix-2 stuff
- 2.4.21-openMosix-1 stuff
- 2.4.22-openMosix-2 stuff
Utilities

qps by Mattias Engdegard is a process manager that can be compiled with Mosix support.
MosixScan by Raphaël Armati is a mosix manager, that allow an easy configuration and monitoring of a mosix cluster, including or not ltsp-mosix.
Quick hack for openmosix of the previous two utilities
- qps-1.9.7-openmosix.tar.gz
- mosixscan-0.2.1-openmosix.tar.gz

What's this?

Mosix is a patch to the linux kernel, that allow process migration. It can be considered in some matter as transparent parallelism. Ltsp (Linux Terminal Server Project) allow a very easy setup of Xterminals.

Merging these two projects allow a very easy setup of a Mosix cluster by using either Xterminal or by temporary turning Windows boxes into diskless Xterminals.

History : We use ltsp in our laboratory for two purposes : first we need real Xterminals to connect to Linux or proprietary Unix servers, second it is a convenient method to turn a Windows box into an xterminal. We also have an office server (StarOffice + Netscape) for students. By the fact we did not have old 486 or MMX to use as Xterminals, so we had to buy diskless new computers for our Xterminals. I then see that the cheapest processor was an Athlon 850 (June 2001) and that the less possible memory was 64 Mo. What a pity to have an Athlon 850 only used for X! So the idea is to use the xterminals as Mosix nodes ... and it works!

Why?

some different reasons to choose this ltsp-mosix solution :

There are a lot of windows machines on your network, with good processors and memory that can be used at night for computation. With a simple floppy disk you can turn the windows computer into a Mosix node.
There is already a ltsp server on your network, and some Xterminals with CPU doing nearly nothing!
Ease of use : once your server is setup, it will only take 5 minutes to add a new node.

How?

Requirements

network : it has to be (at least) a 100Mbits network, for good performances.
Server :Most important point, it has to have a fast hard drive optionaly a lot of memory and a big CPU!
Xterminals/Nodes :a good CPU and sufficient memory.

Setup

On the server you must have a linux distribution installed supported by ltsp, all major distributions are supported.

install LTSP on the server, read the documentation on ltsp site to do that. When it works, that is there is at least one terminal running, go to next step.
install mosix on the server, create a directory /mfs and add the following line to your /etc/fstab
cluster /mfs mfs dfsa=1 0 0
to take advantage of dfsa.
install the ltsp_mosix package
make needed configurations
It should work... mail me if not

FAQ

Q:Why is that page so ugly?
A:Humm do not have time for cosmetics, and I'm not very good for that, but if there is a volonteer he will be welcomed.

Q:I've setup everything and when the ws boot there is a message saying "This is not Mosix"
A:This means the kernel that the ws use does not have Mosix patch compiled in. Check your dhcp configuration to see if it point to the right kernel.

Q:Everything's fine, when I run mon I see all the configured nodes, how can I see if it's actually work?
A:Launch ten or more of the following useless perl script and see the evolution of loads with mon
useless.pl
#!/usr/bin/perl
for ($i=0;$i<10000;$i++) { for ($j=0;$j<10000;$j++) { } }

Q:Is it secure to poweroff a mosix-terminal
A:Unfortunatly no, because there is maybe a process running on it, the only way to be nearly sure will be to switch off mosix on the server, and then poweroff the terminal. The best way, but not already available will be to remotly switch off mosix on the terminal, and then power it off. There is still work to do to improve this point.

Q: What about openmosix?
A: It's done now! (Thanks to Bruce Knox for lobbying)

To do ...

What do you think?

Other pages

WOMP! http://womp.sourceforge.net/ : A micro linux distribution focussed on multimedia.