Lab scripting language?

We have a number of pieces of equipment under computer control,
and wonder whether there exists an interpreted or script language
that could accomplish the control, rather than having to write
and compile C code for every task. This language would have to
be able to link in external C subroutines, as well as hardware
device drivers. We'll probably be mac-based.

People have tossed around names like Python, Cint, and ch, but
I'm hesitant to get involved in something that might generate
even more obscure problems than we have already. The alternative
is to write our own C program which will accept commands at a prompt.
We'd be grateful for suggestions or advice.

Thanks in advance,

rob

In article <1138025107.642432.213220@o13g2000cwo.googlegroups. com>,
rob@haptek.com wrote:

> We have a number of pieces of equipment under computer control,
> and wonder whether there exists an interpreted or script language
> that could accomplish the control, rather than having to write
> and compile C code for every task. This language would have to
> be able to link in external C subroutines, as well as hardware
> device drivers. We'll probably be mac-based.
>
> People have tossed around names like Python, Cint, and ch, but
> I'm hesitant to get involved in something that might generate
> even more obscure problems than we have already. The alternative
> is to write our own C program which will accept commands at a prompt.
> We'd be grateful for suggestions or advice.
>
> Thanks in advance,
>
> rob
>

If you have the money, LabView might be the tool for you. It is pretty
cool. Check out http://www.ni.com/mac/labview.htm and related sites.

-- Lou Pecora (my views are my own) REMOVE THIS to email me.

Look at Lua (www.lua.org) a very simple (seeming) scripting
language that is dead simple to extend, also is free and has a very
liberal license. The manual is quite short and the language is roughly
similar to Pascal. You can have someone doing scripts very quickly but
the language has all the support for objects, structures and other
support for quite complex programming. The language is written in very
portable C and comes as source (just type make). I have not had
problems building it on Linux, Solaris,OS X or embedded systems, they
have done an execllent job of making is very compliant and easy to
compile.

In addition if you need some special functions, it is very very
simple to add new classes for say hardware or some special software
function.

If you are ever tempted to put scripting into a program, run don't
walk to www.lua.org and use it. The language can either be the outside
controller or embedded into a program as the script engine. They do
all the stuff you need and many things you haven't thought of yet. As
an example see the terminal server for OS X (TCP/IP <-> serial port
connector) that is scripted in Lua.
(http://members.bellatlantic.net/~vz.../Load/TERMS.zip) In this case
Lua is the script engine inside the program.

There is also an execllent book written by the authors of the
language (Programming in Lua by Roberto Ierusalimschy).
(http://www.amazon.com/gp/product/85...glance&n=283155)

rob@haptek.com wrote:
> We have a number of pieces of equipment under computer control,
> and wonder whether there exists an interpreted or script language
> that could accomplish the control, rather than having to write
> and compile C code for every task. This language would have to
> be able to link in external C subroutines, as well as hardware
> device drivers. We'll probably be mac-based.

I recommend Tcl. It comes with tiger, is mature, stable and actively
developed, comes with a very easy to use GUI toolkit (way more
cross-platform that Java could ever hope to be), and it is trivial to
link with custom C code (even inlined!).

http://www.tcl.tk is one starting point.

Tcl also has an incredible technology for deployment if that's
something you're concerned about. Similar to mac bundles, you can
combine a tcl/tk runtime, your scripts, associated data, and even an
embedded db into a single executable file. Check out starkits here:
http://www.equi4.com/starkit.html

For adding inline code to your Tcl scripts, check out critcl:
http://www.equi4.com/critcl.html

SWIG, one of the technologies that makes it trivial to link in custom C
code with your tcl scripts, also works with other scripting languages.
Check out SWIG here: http://www.swig.org/.


--
Bryan Oakley
http://www.tclscripting.com

vze35xda@verizon.net wrote:
> Look at Lua (www.lua.org) a very simple (seeming) scripting
> language that is dead simple to extend,

I would also recommend Lua (I've used it in embedded in moderately
complex C++ applications). Its command line interpreter supports
interactive use, or scripts can be compiled to bytecode first.

Thanks for the responses, it's been interesting learning a little about
scripting languages with external C interfaces. We were able to get
a tentative solution to our problem, but not instantly, here are some
notes which might save someone some time. Our answer for the
moment is Lua, Tolua, and Swig.

As a post mentioned, there are dozens of scripting languages and
interpreters, many of which have some kind of external C facility.
There are endless theological discussions about which is best, and
now I have a set of brand-new biases. Based on the advice of another
post ("run, don't walk"!), we looked into Lua. It's an elegantly slim
download, with a remarkable design, and with some hope we looked
into the external C call facility. However it requires C calls of a
specific form, to enable communication with Lua's stack. This is
infeasible for us, as we have a lot of pre-existing routines, graphics,
altivec modules, etc. that we don't want to change, as well as
libraries from third-party vendors for camera control, etc. that we
can't change.

Every language we looked at had some rigidity like that. Maybe we were
asking for too much, no portable scripting language can exist which can
directly call out to raw C routines, due to differences in C calling
conventions between different hardware and platforms. But we thought
something must be possible, as gdb almost does what we want. In gdb
you can "print" some subroutine, and it will be executed in an
interpreted fashion. But there is no built-in looping facility for
example.

Pursuing the C interface issue with lua, we finally found Tolua, and
Swig, which solve the problem in the following way.
Say you have a library of routines, you don't have the source, but you
do have the header files describing what the routines expect for
arguments and return values. You submit the header files as input to
a program which produces, as its output, source code for another
C routine. This code is in the correct format required by Lua,
and it deals with the stack, and does variable translations, and then
calls your library routine. You compile and link in this machine-
generated code, Lua talks to it, and both Lua and the library routine
are happy. There is a little per-call overhead, but probably
negligible in most cases.

As a test I was able to link Lua into a mixed graphics-intensive
program, which writes directly to the frame buffer with SDL,
and links with the carbon, cocoa, and quicktime frameworks.
It worked great, I could stop and start stuff at will, and there
was no perceptible speed penalty. So I guess I'm a convert to
the Cult of Lua, avoid all languages whose names begin with
the letter "p". Incidently the speed penalty for performing
numerics in the interpreter itself isn't too bad, straight C is
maybe 10 times faster for doing typical multiply/accumulate stuff.

I used Tolua to do the interface to C, as it is again slim, you can
read the machine-generated code, and I got it to work. But a friend
of mine also got Swig to work, it has a large user base, and can
produce glue code for a number of scripting languages. He was
able to dynamically link C routines into Lua, even while the
interpreter was running, using code generated by Swig. This,
and Tolua, worked so easily in Lua that we weren't motivated
to try the other interpreters that Swig can service.

We looked a bit at other options, such as C interpreters, as it would
be nice to have C syntax, but without the semi-colons, and we
thought a C interpreter might interact more naturally with C libraries.
But both Cint and ch apparently have their own nontrivial interface
protocols for linking in external C code. Also, they have their own
header files, and we weren't able to get ioctl calls to work, which
we need to run serial ports. Yes, plenty of brand-new equipment
uses good old RS232.

LabView is a possibility, apparently you can link in external C
modules, and there is claimed OSX support. Someone has
managed to call from LabView out to a Lua routine, which then
calls back into LabView. Why, I don't know, but if you can do that,
you probably can do anything. But LabView is expensive, proprietary,
and closed-source, with a top-heavy GUI and an uncertain future.
We already have one of those, it's called a Mac (sorry)!

Thanks for any further suggestions, and sorry for the length of this
post.

rob

ps: about the Tcl post. If I'd seen Tcl first, I'd probably be saying
it was the greatest thing since cheese slices individually wrapped
in plastic, instead of Lua. But Lua has a two-day head start here,
and that may be insurmountable. I like that Lua gave us a prompt
immediately, which is how we'll be using it in the lab. I liked that
Tcl gives you pwd and cd right out of the box. I'm sure each can do
the other in a few lines of code.

Rob Shaw <rob@haptek.com> wrote:

>LabView is a possibility, apparently you can link in external C
>modules, and there is claimed OSX support. Someone has
>managed to call from LabView out to a Lua routine, which then
>calls back into LabView. Why, I don't know, but if you can do that,
>you probably can do anything. But LabView is expensive, proprietary,
>and closed-source, with a top-heavy GUI and an uncertain future.
>We already have one of those, it's called a Mac (sorry)!

Although it is expensive, LabVIEW pays for itself with greatly reduced
development times, including freely available instrument libraries for
a large number of GPIB and serial devices. These libraries alone can
save you tons of development time. LabVIEW's support for OSX isn't
claimed, it is superb and has been available since 10.1.x. LabVIEW was
originally developed on the Macintosh back in the days when the only
way to do data acquisition with PCs was via command lines. The Windows
version didn't appear until Windows 95. The virtual instrument files
are extremely portable between Windows and OSX, and the Full
Development System can compile standalone applications. As their
flagship software product, National Instruments' support for LabVIEW is
quite good.

I'm not sure what you mean by "top-heavy GUI", but the front panel
objects are linked with the graphical programming language in an
intimate way. This intimacy is part of what makes LabVIEW so easy to
program with, and for measurement or automation operators to use.

As for external code, these are done with what are called Code
Interface Nodes (CIN). If the object code follows C linkage, it can be
used in LabVIEW. There are full development headers supplied for CINs.

Rob Shaw wrote:
> Every language we looked at had some rigidity like that. Maybe we were
> asking for too much, no portable scripting language can exist which can
> directly call out to raw C routines, due to differences in C calling
> conventions between different hardware and platforms.

Have you looked into a library called "ffidl"? It allows you from Tcl
(not sure about other languages...) to directly call functions in shared
libraries without having to create wrappers.

Here's an example that shows how to set the application menu name,
calling CPSSetProcessName directly from the scripting environment (I'm
sure wrapping will mangle it horribly...):

::ffidl::callout CPSSetProcessName {pointer-byte pointer-utf8} sint32 \
[::ffidl::symbol
/System/Library/Frameworks/ApplicationServices.framework/Frameworks/CoreGraphics.framework/CoreGraphics
CPSSetProcessName]
CPSSetProcessName [binary format I2 {0 2}] "MyCoolApp"

Other MacOSX-specific examples, including setting the dock icon at
runtime, is on the same page as the above example at
http://wiki.tcl.tk/1197.


I don't want to beat a dead horse if you've already chosen a language,
it's just that I forgot to mention ffidl in my earlier recommendation to
look at Tcl, and ffidl seems to meet the criteria of "directly calling
out to raw C routines".



--
Bryan Oakley
http://www.tclscripting.com

I apologize about my flip comments on LabView, I was speaking out of
ignorance, yet again. They were based on an uninformed emotional
reaction to the marketing web pages, by someone who disapproves
of anything fancier than a command line.
Maybe we should take another look, but let me explain why large
LabView-type applications are probably not the best for us.

We are going to look through a microscope, do image processing,
and depending on what we see, move the stage, or adjust other
parameters, in real time. One route would be to buy packages to do
image tracking, which exist, at least for windows, and perhaps
bolt them onto LabView. We would spend lots of money on software,
but probably be up and running shortly. But the first time we try
and do something which isn't in the pull-down menu, things will get
a lot harder, if not impossible. We expect to be pushing limits of
image processing and device control, and we will have to know
in detail what the code is doing. This might require re-inventing
a few wheels, but we'll be able to quickly change tires.

Then there's the whole Mac issue. You might ask, and indeed I ask
myself often, if we're writing much of our own code, why aren't we
on a linux box, where the source is open right down to the driver
level, and you can get lots of help? Or why not a windows box,
where at least you have the option of chickening out and getting
a software package that might work? One answer is habit,
and an irrational attachment to Macs.

But another is that altivec PPC's are the fastest in their class,
for doing single-precision floating point, by a wide margin, despite
apple's current marketing claims. Just check the FFT benchmarks.
Trying to get close to the hardware for ultimate video performance
can be very frustrating on a mac, things are undocumented, and
subject to change. But I've fought some of those battles, and the
altivec battle, and the speed is what we need, so that argues for mac.

So an extreme statement of the options would be, use canned software
on a windows box which will run immediately, or, write loads of
custom software, orchestrated by a Brazilian dynamic language,
running on a truly fast box, which has been declared obsolete by its
manufacturer.
The choice is clear!

rob

In article <1138735836.351728.199930@g43g2000cwa.googlegroups. com>,
"Rob Shaw" <rob@haptek.com> wrote:

> Then there's the whole Mac issue. You might ask, and indeed I ask
> myself often, if we're writing much of our own code, why aren't we
> on a linux box, where the source is open right down to the driver
> level, and you can get lots of help?

I have found that you can get quite a bit of help on the various Apple
mailing lists for Apple-specific technology, and quite a bit on mailing
lists for open source tech. (For me, most of my open source code comes
from Apache.)

LabView, though, does do a lot of things, and can greatly simplify your
life. The drivers for various bits of hardware are very keen, IME. A
number of our biotech clients swear by it for equipment automation.

> But another is that altivec PPC's are the fastest in their class,
> for doing single-precision floating point, by a wide margin, despite
> apple's current marketing claims. Just check the FFT benchmarks.
> Trying to get close to the hardware for ultimate video performance
> can be very frustrating on a mac, things are undocumented, and
> subject to change. But I've fought some of those battles, and the
> altivec battle, and the speed is what we need, so that argues for mac.

Apple's marketing claims do match your experience. Now that they are on
Intel, you will note that they are discussing spec, esp. SPECint, which
Intel has classically been good at. If Intel gets the speed up enough,
they will win the fp battle too, but it is not clear how well they will
do at that.

For me, having dual core laptops is just plain keen.

> So an extreme statement of the options would be, use canned software
> on a windows box which will run immediately, or, write loads of
> custom software, orchestrated by a Brazilian dynamic language,
> running on a truly fast box, which has been declared obsolete by its
> manufacturer.

There is option three. Use a mix of canned and open source software as
needed to solve your problem, and use the platform that produces the
most happiness and productivity.

For me, that is a Mac, as I really want configure/make to work, but
Linux does not have the productivity apps I want. It may not _stay_
true, but it is certainly true for now.

Scott

--
Scott Ellsworth
scott@alodar.nospam.com
Java and database consulting for the life sciences

Rob Shaw <rob@haptek.com> wrote:

> We are going to look through a microscope, do image processing,
> and depending on what we see, move the stage, or adjust other
> parameters, in real time.

Sounds like you want to do automated cytology, as in PAP smears.
Good luck! Been there, got the T-shirt. Technology can't keep up with
falling market barriers and influx of cheap labor (or does your PPC +
software + peripherals cover its costs for little more than the
equivalent of a bowl of rice/day?)

--
madiba

On 20/08/2006, madiba wrote in message
<1hkdgwu.6akl7ukn6b9cN%down@thekraal.com>:

> Rob Shaw <rob@haptek.com> wrote:
>
> > We are going to look through a microscope, do image processing,
> > and depending on what we see, move the stage, or adjust other
> > parameters, in real time.

What you want is probably MatLab. It can do all the image-processing you
want, and it can be used to control an IO card which could make the
adjustments to the microscope you need. You will need at least one
electronics guy to hook up the microscope and one programmer to program
MatLab.

> Sounds like you want to do automated cytology, as in PAP smears.
> Good luck! Been there, got the T-shirt. Technology can't keep up with
> falling market barriers and influx of cheap labor (or does your PPC +
> software + peripherals cover its costs for little more than the
> equivalent of a bowl of rice/day?)

I think madiba is probably right for a large-scale project with many
performances of the same task. You'd need a real microscope+computer
setup to do this. It's cheaper to offshore it. However, if you can
develop your idea to the point where the hardware becomes cheaper, it
might be a player.

Simon.
--
http://www.hearsay.demon.co.uk

Thanks for the advice. It's certainly true that it's tough to beat the
human eye in image processing. And as long as there are more
people than jobs in the third world, robotics in general is not going
to be cost-effective.

But we're doing experiments that generate dynamical patterns,
it's not a production thing, at least not yet. I'm the electronics
guy, and help with the image processing software, we have a
more-or-less working system.

We ended up using lua as a scripting language, though still we mostly
write in C, it's maybe just as convenient with today's fast compilers.
If I were to do it again, I might look at something called "ipython",
which supposedly has a very nice shell interface, logging facilities,
etc. Lua doesn't have this, but it's so small that you can add stuff
in easily, and the user group is very helpful. It's been a time sink,
though, the jury is still out.

A lot of people swear by Matlab, but we're more in the
reinvent-the-wheel school of thought. One thing we've done is
interface a wireless gaming joystick to our setup, so we can
focus the scope, run the stage, change filters, take pictures,
and run other equipment, all from one unit. The mac
we use works well for us, it's a 1.4 GHz dual MMD, one of the last
with a standard PCI slot. I believe the new ones exclusively use
"PCI express", cards for that standard are vvvery expensive,
if they exist at all. Maybe a linux box makes more sense if
you're starting a setup these days, but we had the mac hardware.

Of course, the real test if is we can do any good science.

Thanks again,

rob

Simon Slavin wrote:
> On 20/08/2006, madiba wrote in message
> <1hkdgwu.6akl7ukn6b9cN%down@thekraal.com>:
>
> > Rob Shaw <rob@haptek.com> wrote:
> >
> > > We are going to look through a microscope, do image processing,
> > > and depending on what we see, move the stage, or adjust other
> > > parameters, in real time.
>
> What you want is probably MatLab. It can do all the image-processing you
> want, and it can be used to control an IO card which could make the
> adjustments to the microscope you need. You will need at least one
> electronics guy to hook up the microscope and one programmer to program
> MatLab.
>
> > Sounds like you want to do automated cytology, as in PAP smears.
> > Good luck! Been there, got the T-shirt. Technology can't keep up with
> > falling market barriers and influx of cheap labor (or does your PPC +
> > software + peripherals cover its costs for little more than the
> > equivalent of a bowl of rice/day?)
>
> I think madiba is probably right for a large-scale project with many
> performances of the same task. You'd need a real microscope+computer
> setup to do this. It's cheaper to offshore it. However, if you can
> develop your idea to the point where the hardware becomes cheaper, it
> might be a player.
>
> Simon.
> --
> http://www.hearsay.demon.co.uk

Give this a shot:

http://homepages.cwi.nl/~jack/macpython/

You may also want to think about Pooch in order to take advantage of
distributed computing and OS X's built in xgrid.


In article <1156517355.559959.55500@75g2000cwc.googlegroups.co m>,
"Rob Shaw" <rob@haptek.com> wrote:

> Thanks for the advice. It's certainly true that it's tough to beat the
> human eye in image processing. And as long as there are more
> people than jobs in the third world, robotics in general is not going
> to be cost-effective.
>
> But we're doing experiments that generate dynamical patterns,
> it's not a production thing, at least not yet. I'm the electronics
> guy, and help with the image processing software, we have a
> more-or-less working system.
>
> We ended up using lua as a scripting language, though still we mostly
> write in C, it's maybe just as convenient with today's fast compilers.
> If I were to do it again, I might look at something called "ipython",
> which supposedly has a very nice shell interface, logging facilities,
> etc. Lua doesn't have this, but it's so small that you can add stuff
> in easily, and the user group is very helpful. It's been a time sink,
> though, the jury is still out.
>
> A lot of people swear by Matlab, but we're more in the
> reinvent-the-wheel school of thought. One thing we've done is
> interface a wireless gaming joystick to our setup, so we can
> focus the scope, run the stage, change filters, take pictures,
> and run other equipment, all from one unit. The mac
> we use works well for us, it's a 1.4 GHz dual MMD, one of the last
> with a standard PCI slot. I believe the new ones exclusively use
> "PCI express", cards for that standard are vvvery expensive,
> if they exist at all. Maybe a linux box makes more sense if
> you're starting a setup these days, but we had the mac hardware.
>
> Of course, the real test if is we can do any good science.
>
> Thanks again,
>
> rob
>
> Simon Slavin wrote:
> > On 20/08/2006, madiba wrote in message
> > <1hkdgwu.6akl7ukn6b9cN%down@thekraal.com>:
> >
> > > Rob Shaw <rob@haptek.com> wrote:
> > >
> > > > We are going to look through a microscope, do image processing,
> > > > and depending on what we see, move the stage, or adjust other
> > > > parameters, in real time.
> >
> > What you want is probably MatLab. It can do all the image-processing you
> > want, and it can be used to control an IO card which could make the
> > adjustments to the microscope you need. You will need at least one
> > electronics guy to hook up the microscope and one programmer to program
> > MatLab.
> >
> > > Sounds like you want to do automated cytology, as in PAP smears.
> > > Good luck! Been there, got the T-shirt. Technology can't keep up with
> > > falling market barriers and influx of cheap labor (or does your PPC +
> > > software + peripherals cover its costs for little more than the
> > > equivalent of a bowl of rice/day?)
> >
> > I think madiba is probably right for a large-scale project with many
> > performances of the same task. You'd need a real microscope+computer
> > setup to do this. It's cheaper to offshore it. However, if you can
> > develop your idea to the point where the hardware becomes cheaper, it
> > might be a player.
> >
> > Simon.
> > --
> > http://www.hearsay.demon.co.uk