Like most developers, I seem to collect old computers. Three such boat anchors have sat in my corner for years. My TODO, to find a computer recycler, never seemed to get done.
This week, quite serendipitously, I stopped by a computer store near a client in downtown Kansas City to inquire about recycling. The store, PC Warehouse, near River City Market, recycles old computers and they will also give your old computer a “hardware makeover.” I elected to do both.
PC Warehouse is the opposite of a big, glistening chain store. It’s a crowded, one-room affair accessible via the side entrance of an old brick building. Inside are stacks of used computers and metal shelves crammed full of plastic bins overflowing with controllers and peripherals scavenged from recycled machines.
At 10 AM I dropped off three aging tower computers and the owner, Dan, helped me select the best candidate for an upgrade. At 2:00 PM I picked up a super fast workstation for a little over $300. The clutter in my corner is gone, two computers were kept out of the landfill, and I got a great “new” development platform.
Dan upgraded my computer with an AMD Phenom X3 (triple core) 64-bit processor on a MSI motherboard sporting the nForce 430 chipset. It has 4GB of RAM, a 500 GB SATA drive, plus a 160 GB SATA drive scavenged from one of the three computers. This added power gives me capacity to try out VMWare Server with multiple guest virtual machines—something I’ll cover in my next blog.
My advice to you is to take those aging computers gathering dust in your corner to a local computer store that recycles PCs, and reward yourself with hardware makeover. It is money well spent.
Thursday, June 18, 2009
Sunday, August 3, 2008
Running Elastic IPs and Ubuntu on Amazon EC2
Background
In my last post, I ran into a reboot problem with Fedora 8, whereby a SSH connection could not be made to a private AMI after it was rebooted. That led me
to test Fedora 4 because it was reported that it did not have the reboot problem. I was able to follow the instructions in my last post almost exactly with the Fedora 4 image; however, there was no mod_proxy_apj available to connect Apache to Tomcat. That meant having to compile and configure mod_jk, which I have done before, but it is a pain I would rather avoid. User "ngasi," who replied to my post on the Amazon Web Services Developers Connection, recommended using the Ubuntu images available from
http://alestic.com. That sounded good to me since my web site runs on Ubuntu.
Getting Started with the Alestic AMI
Start by visiting http://alestic.com/ to see the full list of AMI bundles. I chose ami-26bc584f because that was the recommended Ubuntu bundle on the Alestic site. A detailed description of the bundle is available online.
steven-mitchells-macbook-pro:~ stevecmitchell$ ec2-run-instances ami-26bc584f -k gsg-keypair
RESERVATION r-5363ab3a 123456789012 default
INSTANCE i-f12ef198 ami-26bc584f pending gsg-keypair 0 m1.small 2008-08-03T18:30:19+0000 us-east-1c aki-a71cf9ce ari-a51cf9cc
steven-mitchells-macbook-pro:~ stevecmitchell$ ec2-describe-instances
RESERVATION r-5363ab3a 123456789012 default
INSTANCE i-f12ef198 ami-26bc584f ec2-67-202-53-123.compute-1.amazonaws.com ip-10-251-106-70.ec2.internal running gsg-keypair 0 m1.small 2008-08-03T18:30:19+0000 us-east-1c aki-a71cf9ce ari-a51cf9cc
Steven-mitchells-macbook-pro:~ stevecmitchell$ ssh -i ~/Documents/EC2/id_rsa-gsg-keypair root@ec2-67-202-53-123.compute-1.amazonaws.com
Linux ip-10-251-106-70 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
To access official Ubuntu documentation, please visit:
http://help.ubuntu.com/
Amazon EC2 Ubuntu 8.04 hardy AMI built by
Eric Hammond
For more information: http://ec2hardy.notlong.com
root@ip-10-251-106-70:~#
As with Fedora, I returned to http://www.howtoforge.com for my inspiration, using http://www.howtoforge.com/perfect-server-ubuntu8.04-lts. Please see that page for complete instructions. Here are the highlights from my set-up. I just stuck to what I needed for this test:
apt-get update
apt-get upgrade
apt-get install mysql-server mysql-client libmysqlclient15-dev
apt-get install apache2 apache2-doc apache2-mpm-prefork apache2-utils libexpat1 ssl-cert
apt-get install libapache2-mod-php5 libapache2-mod-ruby php5
php5-common php5-curl php5-dev php5-gd php5-idn php-pear php5-imagick
php5-imap php5-json php5-mcrypt php5-memcache php5-mhash php5-ming
php5-mysql php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy
php5-xmlrpc php5-xsl
Edit /etc/apache2/mods-available/dir.conf and change DirectoryIndex as shown in the Perfect Server HowTo. Next, enabled the necessary Apache modules and toss in a couple more pieces of software:
a2enmod ssl
a2enmod rewrite
a2enmod suexec
a2enmod proxy
a2enmod proxy_ajp
a2enmod proxy_http
a2enmod include
/etc/init.d/apache2 force-reload
apt-get install webalizer
apt-get install ntp ntpdate
We are now ready to install Java and Tomcat. The resource I mainly followed was http://www.howtogeek.com/howto/linux/installing-tomcat-6-on-ubuntu/.
apt-get install sun-java6-jdk
wget http://apache.hoxt.com/tomcat/tomcat-6/v6.0.18/bin/apache-tomcat-6.0.18.tar.gz
tar xvzf apache-tomcat-6.0.18.tar.gz
mv apache-tomcat-6.0.18 /usr/local/tomcat
You need to add export JAVA_HOME=/usr/lib/jvm/java-6-sun. I put it in /usr/local/tomcat/bin/setenv.sh. Other instructions recommended using ~/.bashrc.
Automatic Start-up
Create a start-up script:
vi /etc/init.d/tomcat
Add the following content:
# Tomcat auto-start
#
# description: Auto-starts tomcat (from http://www.howtogeek.com/howto/linux/installing-tomcat-6-on-ubuntu/)
# processname: tomcat
# pidfile: /var/run/tomcat.pid
export JAVA_HOME=/usr/lib/jvm/java-6-sun
case $1 in
start)
sh /usr/local/tomcat/bin/startup.sh
;;
stop)
sh /usr/local/tomcat/bin/shutdown.sh
;;
restart)
sh /usr/local/tomcat/bin/shutdown.sh
sh /usr/local/tomcat/bin/startup.sh
;;
esac
exit 0
Enable and link the start-up script:
sudo chmod 755 /etc/init.d/tomcat
sudo ln -s /etc/init.d/tomcat /etc/rc1.d/K99tomcat
sudo ln -s /etc/init.d/tomcat /etc/rc2.d/S99tomcat
Create a user ID, password, and roles for the Tomcat manager:
vi /usr/local/tomcat/conf/tomcat-users.xml
Edit the contents to your needs:
<tomcat-users>
<role rolename="admin"/>
<role rolename="manager"/>
<user username="myuser" password="mypassword" roles="admin,manager"/>
</tomcat-users>
Apache AJP Proxy Module
Edit proxy.conf to allow traffic from Apache to Tomcat:
vi proxy.conf
You can start by testing the Tomcat Manager and JSP Examples applications. Edit proxy.conf as follows:
<IfModule mod_proxy.c>
#turning ProxyRequests on and allowing proxying from all may allow
#spammers to use your proxy to send email.
ProxyRequests Off
ProxyPreserveHost On
<Proxy *>
AddDefaultCharset off
Order deny,allow
#Deny from all
#Allow from .example.com
</Proxy>
# Enable/disable the handling of HTTP/1.1 "Via:" headers.
# ("Full" adds the server version; "Block" removes all outgoing Via: headers)
# Set to one of: Off | On | Full | Block
ProxyVia On
ProxyPass /tomcat/ ajp://localhost:8009/
ProxyPass /examples/ ajp://localhost:8009/examples/
</IfModule>
You are now ready to start everything and hit it in a browser:
/etc/init.d/tomcat start
/etc/init.d/apache2 force-reload
Use the following URLs for testing, substituting the host name of your EC2 instance:
- http://ec2-67-202-56-6.compute-1.amazonaws.com/tomcat/index.jsp
- http://ec2-67-202-56-6.compute-1.amazonaws.com/examples/index.htm
If something doesn't work right look at /var/log/apache2 or /usr/local/tomcat/logs/. Consider changing /usr/local/tomcat/logs/ to a symbolic link pointing to /var/log/tomcat.
Assuming everything is working, create a custom bundle following these instructions: http://docs.amazonwebservices.com/AWSEC2/2008-02-01/GettingStartedGuide/index.html?creating-an-image.html. I started and rebooted an instance launched from my new custom, private Ubuntu 8.04 AMI to verify it did not experience the re-boot issues I had with Fedora 8 private AMIs. The Ubuntu AMI worked normally, as did Fedora 4.
Since you are probably part of a team it is helpful to share you AMI image with others. This document describes how to share your AMI: http://developer.
steven-mitchells-macbook-pro:~ stevecmitchell$ ec2-modify-image-attribute ami-139f7b7a -l -a 210987654321
steven-mitchells-macbook-pro:~ stevecmitchell$ ec2-describe-image-attribute ami-139f7b7a -l
launchPermission ami-139f7b7a userId 210987654321
launchPermission ami-139f7b7a userId 123456789012
Elastic IP Address
We are now ready to set-up an elastic IP address. Your Amazon EC2 account comes with up to 5 elastic IP address automatically. If you need more than 5 IP addresses you must request additional Elastic IP address from Amazon. The process takes 3-5 days. A details overview of Elastic IP address can be found here, http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1346. It defines elastic IPs as follows:
"Elastic IP addresses are static IP addresses designed for dynamic cloud computing.
An Elastic IP address is associated with your account, not a particular instance,
and you control that address until you choose to explicitly release it. Unlike
traditional static IP addresses, however, Elastic IP addresses allow you to
mask instance or availability zone failures by programmatically remapping your
public IP addresses to any instance associated with your account. Rather than
waiting on a data technician to reconfigure or replace your host, or waiting
for DNS to propagate to all of your customers, Amazon EC2 enables you to engineer
around problems with your instance or software by programmatically remapping
your Elastic IP address to a replacement instance. "
There process is very simple, as shown below:
steven-mitchells-macbook-pro:~ stevecmitchell$ ec2-allocate-address
ADDRESS 77.102.143.105
steven-mitchells-macbook-pro:~ stevecmitchell$ ec2-describe-addresses
ADDRESS 77.102.143.105 <-- Note that it is not assigned to an instance yet
steven-mitchells-macbook-pro:~ stevecmitchell$ ec2-describe-instances
RESERVATION r-eb7bb382 123456789012 default
INSTANCE i-f12ef198 ami-279e7a4e ec2-75-101-194-69.compute-1.amazonaws.com ip-10-251-198-68.ec2.internal running gsg-keypair 0 m1.small 2008-08-04T00:15:49+0000 us-east-1c aki-a71cf9ce ari-a51cf9cc
steven-mitchells-macbook-pro:~ stevecmitchell$ ec2-associate-address -i i-f12ef198 77.102.143.105
ADDRESS 77.102.143.105 i-f12ef198
steven-mitchells-macbook-pro:~ stevecmitchell$ ssh -i id_rsa-gsg-keypair root@77.102.143.105 <-- It may be a couple of minutes before this works.
That is all there is to it. When it is time to switch to a different instance you can use ec2-disassociate-address to disassociate the elastic IP and then assign it to a new instance. That is a very powerful feature.
Caveat: Amazon AMI instances can have only one elastic IP assigned at a time. That means you cannot host multiple web sites on your instance if you are using separate SSL certificates. Of course, if you use a wildcard cert you can share the SSL cert (www.mydomain.com, dev.mydomain.com, team.mydomain.com, etc...). Multiple elastic IP address per AMI instance is on everybody's wish list.
Conclusion
This Ubuntu AMI was definitely the easiest one for me to use with Tomcat. Kudos to Eric Hammond of Alestic for setting making his AMIs public. I found Linux AMIs easier to use in general than OpenSolaris. I could not connect to my Fedora 8 AMI with SSH after it was rebooted, even after following the tip to remove Kudzu. Fedora 4 did not have the connection issues after a re-boot, but Apache did not come with Mod Proxy AJP on that distribution. I also ran into directory permission issue on Fedora that prevented Tomcat from compiling JSPs (see earlier post for resolution). Eric Hammond's Ubuntu 8.04 AMI had none of these issues. I simply followed the steps in this post and got the system running the way I wanted.
Friday, August 1, 2008
Running Fedora 8, MySQL, and Tomcat on Amazon EC2
Cloud Computing
See my previous post, Running OpenSolaris, MySql, and Glassfish, for instructions about setting up Amazon EC2 and starting an AMI instance. This post is continues that theme with a Linux AMI.What I'm Not Telling You
Follow this post and you will have an running Tomcat instance in the Amazon cloud when you are through. It actually was not that simple to arrive at this point. I fought a Tomcat permission problem for hours (not related to EC2), mistakenly deleted the S3 files for my "good" checkpoint private instance along the way, found myself with an instance that I could not ssh into after creating it, and had one image that could not be bundled. On the bright side, I had the opportunity to validate these instructions several times. It got pretty routine after awhile. I also became much more familiar with commands, like ec2-deregister, than I otherwise would have become. The main lesson I learned was to stop and create a new private image after each major change, and to use ec2-reboot-instances periodically to make sure your image is bootable before taking the time to make new private images. If at any point you find you have buggered it up a bit, just use ec2-terminate-instances and then fallback to your last good image.Picking an AMI Image
In my last post, I dipped my toes in the Amazon EC2 waters by testing the Sun OpenSolaris EC2 (AMI) Amazon Machine Image. In this post I build on that experience by deploying Tomcat on Amazon EC2. The first decision to make is what AMI image to use. As a developer, I prefer the simplicity of Tomcat running on Linux.There are many public AMI images posted in the Amazon Web Services Developer Connection. These public AMIs may also be accessed via Amazon Web services, as I have done below in combination with the Linux grep command:
steven-mitchells-macbook-pro:EC2 stevecmitchell$ ec2dim -x all | grep tomcatIn the example above I "grepped" for AMI images that included the word "tomcat." Only two images were found, ami-443bde2d, which is no longer supported, and ami-4599c2c, which is a paid AMI (I found not information about who offers that paid AMI). You can see it is a paid AMI by the presence of the Product Code, BA7154BF.
IMAGE ami-443bde2d appservers/jee/tomcat.manifest.xml 734463992416 available public i386 machine
IMAGE ami-45997c2c public-images/fc6-tomcat-java.manifest.xml 365812321051 available public BA7154BF i386 machine
Ultimately, my client will need a multi-tier environment with a presentation tier (Apache), an application tier (Tomcat), and a database tier (MySql). My needs are much simpler as I only need to provide a proof-of-concept demonstration environment and a new home for my web site, www.byteworksinc.com. I can get by with a single node combined tier. When the time comes for a 3-tier dozen-node environment we can get the help of vendor like RightScale.
Another consideration is the source of AMI image. Anybody can post a public image. I chose the more conservative approach of only looking for images owned by Amazon. Let's try that search again limited to Amazon and MySql:
steven-mitchells-macbook-pro:EC2 stevecmitchell$ ec2dim -o amazon | grep mysqlUnfortunately, all the images with MySQL pre-installed are Fedora 4. Finally, I opened up my search to all Amazon-owned images and picked ami-2b5fba42, a "naked" Fedora 8 image.
IMAGE ami-225fba4b ec2-public-images/fedora-core4-apache-mysql-v1.07.manifest.xml amazon available public i386 machine
IMAGE ami-25b6534c ec2-public-images/fedora-core4-apache-mysql.manifest.xml amazon available public i386 machine
IMAGE ami-255fba4c ec2-public-images/fedora-core4-mysql-v1.07.manifest.xml amazon available public i386 machine
IMAGE ami-22b6534b ec2-public-images/fedora-core4-mysql.manifest.xml amazon available public i386 machine
steven-mitchells-macbook-pro:EC2 stevecmitchell$ ec2-run-instances ami-2b5fba42 -k gsg-keypairNext, I installed the systems I would need using http://www.howtoforge.com/perfect-server-fedora9 for inspiration.
RESERVATION r-4ee62e27 12345678901 default
INSTANCE i-8a9c43e3 ami-2b5fba42 pending gsg-keypair 0 m1.small 2008-07-31T12:53:45+0000 us-east-1b aki-a71cf9ce ari-a51cf9cc
steven-mitchells-macbook-pro:EC2 stevecmitchell$ ec2-describe-instances
RESERVATION r-4ee62e27 12345678901 default
INSTANCE i-8a9c43e3 ami-2b5fba42 ec2-75-101-216-54.compute-1.amazonaws.com domU-12-31-38-01-BD-F4.compute-1.internal running gsg-keypair 0 m1.small 2008-07-31T12:53:45+0000 us-east-1b aki-a71cf9ce ari-a51cf9cc
steven-mitchells-macbook-pro:EC2 stevecmitchell$ ssh -i id_rsa-gsg-keypair root@ec2-75-101-216-54.compute-1.amazonaws.com
__| __|_ ) Fedora 8
_| ( / 32-bit
___|___|___|
Welcome to an EC2 Public Image
:-)
Base
--[ see /etc/ec2/release-notes ]--
TIP - For your first pass, stick to the commands shown below. When I installed everything (almost) from perfect-server-fedora9 I was left with an image that could not be bundled. It crashed one of the rsync commands called by ec2-bundle-vol. You can always come back and add more software after you have created a "checkpoint" private AMI to fallback to in case of problems.
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY*To complete the php install you need to edit /etc/httpd/conf/httpd.conf and change the DirectoryIndex as shown below:
yum update
Dependencies Resolved
=============================================================================
Package Arch Version Repository Size
=============================================================================
Installing:
kernel-xen i686 2.6.21.7-3.fc8 updates 17 M
Updating:
...
Transaction Summary
=============================================================================
Install 4 Package(s)
Update 89 Package(s)
Remove 0 Package(s)
Total download size: 95 M
Is this ok [y/N]:
yum -y install mysql mysql-devel mysql-server
chkconfig --levels 235 mysqld on
/etc/init.d/mysqld start
netstat -tap | grep mysql
mysqladmin -u root password yourrootsqlpassword
yum install php php-devel php-gd php-imap php-ldap php-mysql php-odbc php-pear php-xml php-xmlrpc php-eaccelerator php-magickwand php-magpierss php-mapserver php-mbstring php-mcrypt php-mhash php-mssql
php-shout php-snmp php-soap php-tidy curl curl-devel perl-libwww-perl ImageMagick libxml2 libxml2-devel
DirectoryIndex index.html index.htm index.shtml index.cgi index.php index.php3 index.pl
After editing httpd.conf start Apache:
chkconfig --levels 235 httpd onOnce you start httpd you can test it using your AMI instance URL (e.g. http://ec2-75-101-216-54.compute-1.amazonaws.com/).
/etc/init.d/httpd start
Next, I set about installing Java and Tomcat:
wget http://www.java.net/download/jdk6/6u10/promoted/b28/binaries/jdk-6u10-rc-bin-b28-linux-i586-21_jul_2008-rpm.binFind the location of the JVM so the alternatives can be set:
chmod 775 jdk-6u10-rc-bin-b28-linux-i586-21_jul_2008-rpm.bin?e=1217511340&h=86dd80d26f9cf05001fb7217f3959b7d
./jdk-6u10-rc-bin-b28-linux-i586-21_jul_2008-rpm.bin?e=1217511340&h=86dd80d26f9cf05001fb7217f3959b7d
[root@domU-12-31-38-01-BD-F4 ~]# java -version
java version "1.6.0_10-rc"
Java(TM) SE Runtime Environment (build 1.6.0_10-rc-b28)
Java HotSpot(TM) Client VM (build 11.0-b15, mixed mode, sharing)
[root@domU-12-31-38-01-BD-F4 ~]# updatedb;locate javac |grep binUse that location to set-up the alternatives:
/usr/bin/javac
/usr/java/jdk1.6.0_10/bin/javac
You are now ready to install Tomcat:alternatives --install /usr/bin/java java/usr/java/jdk1.6.0_10/bin/java 100alternatives --install /usr/bin/jar jar/usr/java/jdk1.6.0_10/bin/jar 100alternatives --install /usr/bin/javac javac/usr/java/jdk1.6.0_10/bin/javac 100
[Verify the setting...]
[root@domU-12-31-38-01-BD-F4 ~]# /usr/sbin/alternatives --config java
There is 1 program that provides 'java'.
Selection Command
-----------------------------------------------
*+ 1 /usr/java/jdk1.6.0_10/bin/java
yum install tomcat5 tomcat5-webapps tomcat5-admin-webappsEdit the Apache config to enable the mod_proxy_ajp:
[root@domU-12-31-38-01-BD-F4 ~]# vi /etc/httpd/conf.d/proxy_ajp.confOne last chore, you will need to edit the /etc/tomcat5/tomcat-users.xml and set-up a user for the Tomcat Manager application if you plan to use it:
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
#
# When loaded, the mod_proxy_ajp module adds support for
# proxying to an AJP/1.3 backend server (such as Tomcat).
# To proxy to an AJP backend, use the "ajp://" URI scheme;
# Tomcat is configured to listen on port 8009 for AJP requests
# by default.
#
#
# Uncomment the following lines to serve the ROOT webapp
# under the /tomcat/ location, and the jsp-examples webapp
# under the /examples/ location.
#
ProxyPass /tomcat/ ajp://localhost:8009/
ProxyPass /examples/ ajp://localhost:8009/jsp-examples/
<?xml version='1.0' encoding='utf-8'?>You are now ready to test Tomcat:
<tomcat-users>
<role rolename="admin"/>
<role rolename="manager"/>
<user username="tomcat" password="myverysecurepassword" roles="admin,manager"/>
</tomcat-users>
[root@domU-12-31-38-01-BD-F4 httpd]# /etc/init.d/tomcat5 startTest your server with this URLS (substituting your instance URL):
Starting tomcat5: [ OK ]
[root@domU-12-31-38-01-BD-F4 httpd]# /etc/init.d/httpd restart
Stopping httpd: [ OK ]
Starting httpd: [Fri Aug 01 06:49:11 2008] [warn] worker ajp://localhost:8009/ already used by another worker
- Tomcat Examples - http://ec2-75-101-216-54.compute-1.amazonaws.com/examples/index.html
- Tomcat Manager - http://ec2-75-101-216-54.compute-1.amazonaws.com/tomcat/index.jsp
If you run into problems have a look under /var/logs/httpd or /var/logs/tomcat5.
Caution - Although the links above were successful, the JSPs in the first applications I deployed to EC2 would not compile. The problem turned out to be file permission issues that I solved following this post: http://saloon.javaranch.com/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic&f=56&t=006169
Now that you've invested time an energy getting your instance to work to your liking, its time to back it up. Remember, cloud computing is ephemeral, so if you terminate your instance so you don't leave the meter running, then all your work will be lost!
TIP - Before creating a custom image verify that your instance is rebootable, in so far as you are able to connect to it again after a reboot. See this post about reboot problems with Fedora 8: http://developer.amazonwebservices.com/connect/thread.jspa?messageID=57056. Before attempting a reboot you want to remove kudzu with the command shown below:
steven-mitchells-macbook-pro:~ stevecmitchell$ chkconfig --del kudzuTo create a custom bundle and put it in your S3 account, following these instructions: http://docs.amazonwebservices.com/AWSEC2/2008-02-01/GettingStartedGuide/index.html?creating-an-image.html
steven-mitchells-macbook-pro:~ stevecmitchell$ ec2-reboot-instances i-e911ce8c
steven-mitchells-macbook-pro:~ stevecmitchell$ ssh -i ~/Documents/EC2/id_rsa-gsg-keypair root@ec2-75-101-209-255.compute-1.amazonaws.com
Last login: Sat Aug 2 19:23:30 2008 from 98.151.31.151
__| __|_ ) Fedora 8
_| ( / 32-bit
___|___|___|
Welcome to an EC2 Public Image
I had a problem with my connection timing out. Though I am sure there is a better way to get around it, I used the ampersand at the end of the command to run the command in the background, and
ec2-bundle-vol -d /mnt -k /mnt/pk-123456[...].pem -c /mnt/cert-123456[...].pem -u 123456789012 -r i386 -p byteworks002 &then I used the up arrow to re-issue the "ps -ael | grep ec2" command every couple of minutes to keep the connection alive. In the time I spent doing that I could have found a work around, I'm sure. Once that command finished it was a simple matter of following the remaining instructions to copy the parts to S3 and then register a private AMI. Luckily, my connection did not timeout uploading my image to S3 or registering my private AMI with Amazon. The following command shows you a list of your registered images:
[1] 6302
[root@ip-10-251-198-116 mnt]# Copying / into the image file /mnt/byteworks002...
Excluding:
/sys
/proc
/sys/fs/fuse/connections
/proc/sys/fs/binfmt_misc
/dev
/media
/mnt
/proc
/sys
/mnt/byteworks002
/mnt/img-mnt
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.002673 s, 392 MB/s
mke2fs 1.40.4 (31-Dec-2007)
[root@ip-10-251-198-116 mnt]# NOTE: rsync with preservation of extended file attributes failed. Retrying rsync
without attempting to preserve extended file attributes...
[root@ip-10-251-198-116 mnt]# ps -ael | grep ec2
0 S 0 6302 6227 0 78 0 - 647 wait pts/1 00:00:00 ec2-bundle-vol
steven-mitchells-macbook-pro:~ stevecmitchell$ ec2-describe-imagesCAUTION - Your private AMI is just a pointer to the files stored on S3. If you delete those files from S3 the image will be unusable.
IMAGE ami-569d793f byteworks-00001/byteworks001.manifest.xml 12345678901 available private i386 machine aki-a71cf9ce ari-a51cf9cc
IMAGE ami-dd9d79b4 byteworks-00002/byteworks002.manifest.xml 12345678901 available private i386 machine aki-a71cf9ce ari-a51cf9cc
Closing Comments
I was not able to overcome the reboot issue with Fedora 8. Even after removing Kudzu from my images I could not connect to my instances after a reboot. That was not a show stopper for experimenting with EC2. If you cannot connect after a reboot... don't reboot, just terminate your instance and relaunch it. That is not an acceptable strategy for the long-term, so my search for the best Java AMI continues. I swore I wasn't going to use anything other than Tomcat, but maybe I should look at the JBoss AMI? Perhaps I will eventually, but first I'll see if I can get Tomcat running the way I like.
Wednesday, July 30, 2008
Running OpenSolaris, MySQL, and Glassfish on Amazon EC2
Cloud Computing
Google "Cloud Computing" and you will find hundreds of thousands of hits. The PAAS (Platform as a Service) market has taken off, as covered in Business Week's article "Computing Heads for the Clouds." In this post I document my journey into cloud computing.
My need to explore cloud computing was driven by a client request. My client has a start-up company and wanted a way to quickly deploy a scalable application without a large investment in infrastructure. Cloud computing definitely fits the bill. I think of cloud computing as deploying a VMWare image across the network, only you don't control what specific server it runs on. You simply start-up your instance in the "cloud," get its url, then connect to it like a regular server.
A disadvantage of most cloud computing is that instances are ephemeral, meaning that if you terminate your instance, any new data stored since the instance was loaded is lost. It is your responsibility to have a back-up plan for your data.
If you are a Gartner member, I recommend reading "IT Operational Considerations for Cloud Computing." It tells a cautionary tail of a rapidly growing, emerging market where the doctrine of "caveat emptor" (let the buyer beware) certainly applies. Gartner sees most early offerings suitable for non-mission critical applications, with offerings for mission critical applications not far behind. They stress doing your homework and getting a SLA in place.
Amazon EC2
Amazon EC2 (Elastic Compute Cloud) consists of virtual Linux instances running on XEN. To get an overview of Amazon EC2 have a look Jeff Barr's presentation "Amazon Web Services." Get started with the Amazon EC2 Getting Started Guide, where you will create an Amazon Web Services account, install the EC2 tools and configure your computer, and start and stop one of the public AMIs (Amazon Machine Image). A very handy tool for Firefox is Elasticfox , an add-on that gives you a graphical interface to manage your running AMI instances, and S3 Firefox, a S3 (Simple Storage Solution) GUI for Firefox that gives you a a GUI file browser. The Firefox plug-ins are options since everything can be done using command line Web services, but the GUI does make it all easier. Other exciting news is that Amazon announced persistent storage back in April. You can apply for the EC2 persistent storage beta program at http://www.amazon.com/gp/html-forms-controller/ec2-persistent-storage.With EC2 you can choose from many public AMI bundles, get a private AMI from a third party, or you can create your own AMI. Sun, RedHat, and other third parties have announces support for Amazon EC2. I chose the OpenSolaris AMI because my client is interested in the support contract available from Sun for their OS, application server, and database. You can get get a similar offering from Redhat.
OpenSolaris AMI
For this post I will be setting up the OpenSolaris AMI announced by Sun in May. You can sign-up for access at http://www.sun.com/third-party/global/amazon/index.jsp. Once you have access just follow the OpenSolaris EC2 Getting Started Guide.For this post the AMI I used was OpenSolaris (SXCE) + GlassFish + MySQL (32-bit AMI: ami-3742a75e aki-b57b9edc ari-b47b9edd / sun-osol/GFAS-MySQL-). Which i started from my EC2 id_rsa-gsg-keypair directory using the following command:
steven-mitchells-macbook-pro:EC2 $ ec2-run-instances ami-3742a75e -k gsg-keypair
RESERVATION r-ccaa68a5 12345678910 default
INSTANCE i-74da0d1d ami-3742a75e pending gsg-keypair 0 m1.small 2008-06-21T19:30:07+0000 us-east-1b aki-b57b9edc ari-b47b9ed
Viewed from ElasticFox, the running instance looks like this (see bottom area of screenshot).
ElasticFox (or the EC ) shows the public DNS assigned to your instance. You can get the same information by using this command:
steven-mitchells-macbook-pro:EC2 $ ec2-describe-instances i-74da0d1d
You need the public URL to SSH into the box, as shown below:
steven-mitchells-macbook-pro:EC2 $ ssh -i id_rsa-gsg-keypair root@ec2-67-202-43-88.compute-1.amazonaws.comInformation about using this beta image is available on Rudolph Kutina's Blog: http://blogs.sun.com/VirtualGuru/entry/pre_build_sxde79_jeos_rnjs#AMICompositeImage:EmptySXDE79JeOSRNJSS3GlassfishV3TP2update. Take time to also look through Rudolph's EC2 references too: "OpenSolaris on EC2 Workshop".
The authenticity of host 'ec2-67-202-43-88.compute-1.amazonaws.com (67.202.43.88)' can't be established.
RSA key fingerprint is bb:27:36:dd:af:eb:9d:8e:48:76:0f:eb:28:c3:45:7e.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ec2-67-202-43-88.compute-1.amazonaws.com,67.202.43.88' (RSA) to the list of known hosts.
Last login: Wed May 7 07:26:14 2008 from 129.157.104.193
-------100------ Welcome to OpenSolaris on Amazon AWS (EC2,S3) Prototypes --------100-------
Image: JeOS RNSP - Solaris Express JeOS with Reduced Network Server Profile SXDE0108 snv_79a
This AMI image is Prof-of-Concept and is presented for evaluation, demostration or learning
purposes only. This image can contain software and/or represent procedures in BETA quality.
JeOS [ Ver: SXDE79a-1.0, Pkgs: 311, Sizes: [Pkgs 876MB, Inst 1.1GB, ZIP 448MG, EC2 462MB] ]
Shells [ sh ksh bash tcsh zsh mc screen ] Editors [ nano joe vim ] Langs [ Perl Ruby Java ]
Archvers [ tar gtar gzip bzip2 zip ] SysTools [ traceroute ntpdate cron top logadm sfMANs ]
ec2ify [ /opt/ec2ify: s3tool ec2ident(SMF) ec2ify(SMF) [info mount params updater syncer] ]
Apps [ Glassfish as Sun Java System Application Server 9.1 Update 1 with MySQL bundle ]
DBs [ MySQL Server 5.0 - Community Edition [ Bundle Sizes 377MB Zip: 150MB ] ]
To list all installed packages use command 'pkginfo', optional software is installed in /opt
For OpenSolaris on Amazon EC2 program http://www.sun.com/third-party/global/amazon/index.jsp
For more info about JeOS creation and more tips look in blog http://blogs.sun.com/VirtualGuru
OS type: solaris Manifest: sun-osol/GFAS-MySQL-79_32_1.0.img.manifest.xml
Type: m1.small Index: 0 i-ID: i-8c19dbe5 AMI: ami-3742a75e AKI: aki-b57b9edc ARI: ari-b47b9edd
Con: ec2-67-202-43-88.compute-1.amazonaws.com->domU-12-31-38-00-A0-E1.compute-1.internal
Con: 67.202.43.88->10.252.167.15 AZ: us-east-1b SG: default KEY: gsg-keypair
root@domU-12-31-38-00-A0-E1
MySQL
Once you have connected to your instance cloud computing is not very different from working with any remote server. If you are running the ami-3742a75e image you start by configuring MySQL. The full details of setting up MySQL are beyond the scope of this post. There are many good on-line references, including this one: http://docs.sun.com/app/docs/doc/820-3797/ggkei?a=view. At a minimum you will have to issue this command from /opt/glassfish/myslq:mysql# sh scripts/mysql_install_db --no-defaults --basedir=/opt/glassfish/mysql --datadir=/opt/glassfish/mysql/dataTo start MySQL you can use this command.
./bin/mysqld_safe --defaults-file=/opt/glassfish/mysql/mysql.ini --user=root &Next, MySQL need to be set-up as a service. BigAdmin has an excellent post on how to do this, http://www.sun.com/bigadmin/content/submitted/mysql_smf_tip.html, that requires a few changes:
- The
/var/svc/manifest/network/mysql.xmlfile can be used exactly as shown in the article. - The MySQL bin and data paths need to be corrected in /lib/svc/method/svc-mysql:
- Change DB_DIR=/var/mysql to DB_DIR=/opt/glassfish/mysql/data
- Change/usr/sfw/sbin/mysqld_safe to /opt/glassfish/mysql/bin/mysqld_safe
After completing the rest of the commands in the article MySQL should be running as a service.
Glassfish
Instructions for starting Glassfish are here: http://blogs.sun.com/VirtualGuru/entry/inst_conf_sgmj_solaris_glassfish. Here is the output from the command. Note that output is sent to /opt/glassfish/domains/domain1/logs/server.log.root@domU-12-31-38-00-A0-E1# /opt/glassfish/bin/asadmin start-domain domain1
Starting Domain domain1, please wait.
Log redirected to /opt/glassfish/domains/domain1/logs/server.log.
Redirecting output to /opt/glassfish/domains/domain1/logs/server.log
Domain domain1 is ready to receive client requests. Additional services are being started in background.
Domain [domain1] is running [Sun Java System Application Server 9.1_01 (build b09d-fcs)] with its configuration and logs at: [/opt/glassfish/domains].
Admin Console is available at [http://localhost:4848].
Use the same port [4848] for "asadmin" commands.
User web applications are available at these URLs:
[http://localhost:8080 https://localhost:8181 ].
Following web-contexts are available:
[/web1 /__wstx-services ].
Standard JMX Clients (like JConsole) can connect to JMXServiceURL:
[service:jmx:rmi:///jndi/rmi://domU-12-31-38-00-A0-E1.compute-1.internal:8686/jmxrmi] for domain management purposes.
Domain listens on at least following ports for connections:
[8080 8181 4848 3700 3820 3920 8686 ].
Domain does not support application server clusters and other standalone instances.
You are now ready to log into the Glassfish admin console at http://ec2-67-202-43-88.compute-1.amazonaws.com:4848 (Be sure to change the server name to match that of your instance).
Sign into the console using the credentials found in Rudolph's blog (check the blog, as they might change). Most likely, you will want to create a connection pool and JDBC resource for your application. That is beyond the scope of this post.
It set-up Glassfish as a SMF service, I followed the instructions found here: http://blogs.sun.com/VirtualGuru/entry/inst_amp_conf_liferay_on
Tune the JVM HEAP memory in /opt/glassfish/domains/domain1/config/domain.xml from 512m to 1024m.
<jvm-option>-Xmx1024m</jvm-option>Create minimal Glassfish SMF script, /opt/glassfish/glassfish.xml, to automatic start and stop Glassfish.
<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<service_bundle type='manifest' name='glassfish:domain1'>
<service name='glassfish/domain1' type='service' version='1'>
<create_default_instance enabled='true' />
<single_instance />
<dependency name='fs-local' grouping='require_all' restart_on='none' type='service'>
<service_fmri value='svc:/system/filesystem/local' />
</dependency>
<dependency name='network-service' grouping='require_all' restart_on='none' type='service'>
<service_fmri value='svc:/network/service' />
</dependency>
<exec_method type='method' name='start' exec='/opt/glassfish/bin/asadmin start-domain domain1' timeout_seconds='300' />
<exec_method type='method' name='stop' exec='/opt/glassfish/bin/asadmin stop-domain domain1' timeout_seconds='150' />
<property_group name='startd' type='framework'>
<propval name='duration' type='astring' value='transient' />
</property_group>
<stability value='Unstable' />
<template>
<common_name>
<loctext xml:lang='C'>Glassfish V2</loctext>
</common_name>
</template>
</service>
</service_bundle>
Load SMF this XML definition
/usr/sbin/svccfg validate /opt/glassfish/glassfish.xml
/usr/sbin/svccfg import /opt/glassfish/glassfish.xml
/usr/sbin/svcadm enable svc:/glassfish/domain1:default
Check then SMF service working OK
svcs -a | grep glassfish
online 21:46:04 svc:/glassfish/domain1:default
svcs -l svc:/glassfish/domain1:default
fmri svc:/glassfish/domain1:default
name Glassfish V2
enabled true
state online
next_state none
state_time Tue Apr 29 21:46:04 2008
logfile /var/svc/log/glassfish-domain1:default.log
restarter svc:/system/svc/restarter:default
dependency require_all/none svc:/system/filesystem/local (online)
dependency require_all/none svc:/network/service (online)
cat /var/svc/log/glassfish-domain1:default.log
Domain [domain1] is running [Sun Java System Application Server 9.1_01 (build b09d-fcs)]
with its configuration and logs at: [/opt/glassfish/domains].
Admin Console is available at [http://localhost:4848].
Use the same port [4848] for "asadmin" commands.
User web applications are available at these URLs:
[http://localhost:8080 https://localhost:8181 ].
Following web-contexts are available:
[/web1 /__wstx-services tunnel-web / liferay-jedi-theme ].
Standard JMX Clients (like JConsole) can connect to JMXServiceURL:
[service:jmx:rmi:///jndi/rmi://domU-12-31-39-00-69-D8.compute-1.internal:8686/jmxrmi] for domain management purposes.
Domain listens on at least following ports for connections:
[8080 8181 4848 3700 3820 3920 8686 ].
Domain does not support application server clusters and other standalone instances.
Conclusion
Following the resources available on the Web it was relatively straightforward to create this AMI instance with MySql and Glassfish running as services. For the moment I have suspended my investigation of the OpenSolaris bundle with Glassfish. Next, I plan to try out a Linux AMI with MySql and Tomcat.Wednesday, February 6, 2008
Return of the Jetty (embedded OSGi Server)
In my last post you learned that I needed to create a server application bundle that could be deployed to an Arcom Zeus device running embedded Linux and Equinox.
Instead of deploying a web application to an application server, like most Java developers are used to, the bundle is deployed to Equinox along side an application server, as shown above. In other words, Jetty is just another bundle running on Equinox.
Creating a OSGi Server Application
Web enabling the application involves creating a document root in which to place the static site content and adding Servlets. First, we create a new folder for the static content and then we declare its location using an extension point. The org.eclipse.equinox.http.registry bundle has an extension point named "resources" that can be used for declaring static content. We need to add that extension to our project and configure it to point to the directory containing the static content. Later in the project we will use the "servlets" extension point of the same bundle to register a Servlet.- Create a folder at the root of your project to hold your static content. I called mine "WebContent."
- Add a simple index.html page to the new folder.
- Enable Extension configuration in the Manifests Wizard by opening the Overview tab and clicking Extensions.
- Click OK to display the Extensions page. This will add an Extensions tab the Manifests wizard that will automatically open.
- Click the Add button and then type "org.eclipse.equinox.http.registry.resources" in the Extension Point Filter field. Be to sure to uncheck "Show only extension points from the required plug-ins" to get the bundle to appear in the filtered list. Select the bundle and then click finish.
- The Manifests wizard will prompt whether to add a dependency to your project for this bundle. Click Yes.
- Add an ID and Name for this extension in your project. I called mine "staticContent" to indicate what I was using putting in the resources directory. You can name yours something else if you like. Save your changes.
- Now you are ready to add a resource definition to the extension. Right click on the bundle and select New --> Resource.
- Now you need to set the Alias (the resource path in the URL), and Base-name, which in our case is the directory "WebContent." Don't worry about Http Context Id. That requires the org.eclipse.equinox.http.registry.httpcontexts extension point, which is outside the scope of this post.
- Display the plugin.xml contents to see what the Manifests Wizard did:
<?xml version="1.0" encoding="UTF-8"?>
<?eclipse version="3.2"?>
<plugin>
<extension
id="staticContent"
name="StaticContent"
point="org.eclipse.equinox.http.registry.resources">
<resource
alias="/">
base-name="WebContent">
</resource>
</extension>
</plugin>
- To better understand the extension values, return to the Extension tab and click the "Open extension point schema" link, and the open the Example tab on the resource.exsd page.
Progress Check - Serving Static HTML
Let's validate that our bundle can serve static HTML content from our OSGi bundle before moving on to Servlets. To do that we need to make some environment changes. We'll create a copy of the OSGi configuration file from my last post first.- Select Run --> Open Run Dialog...
- Highlight the Simple OSGi Configuration that you created in my last post, click the right mouse, and choose Duplicate.
- Change the name to Simple OSGi Servlet Configuration.
- (optional) Select the arguments tab and add -Dorg.osgi.service.http.port to assign the port number. I chose port 8081 for OSGi.
Return to the Bundles tab so that we can select the Jetty dependencies
- In addition to selecting your project, select the following bundles:
- javax.servlet
- org.apache.commons.logging
- org.apache.xerces
- org.apache.xml.resolver
- org.eclipse.core.jobs
- org.eclipse.core.runtime.compatibility.registry
- org.eclipse.equinox.common
- org.eclipse.equinox.http.jetty
- org.eclipse.equinox.http.registry
- org.eclipse.equinox.http.servlet
- org.eclipse.equinox.registry
- org.eclipse.osgi.services
- org.mortbay.jetty
Once your bundle, compiled code, and Equinox OSGi runtime environment are compatible you should see something like this:
osgi> Feb 5, 2008 11:32:01 PM org.mortbay.http.HttpServer doStart
INFO: Version Jetty/5.1.x
Feb 5, 2008 11:32:01 PM org.mortbay.util.Container start
INFO: Started org.mortbay.jetty.servlet.ServletHandler@1a626f
Feb 5, 2008 11:32:01 PM org.mortbay.util.Container start INFO: Started HttpContext[/,/]
Feb 5, 2008 11:32:01 PM org.mortbay.http.SocketListener start
INFO: Started SocketListener on 0.0.0.0:8081
Feb 5, 2008 11:32:01 PM org.mortbay.util.Container start
INFO: Started org.mortbay.http.HttpServer@291aff
MyFirstActivator is starting. Hello.
Now try hitting your URL: http://localhost:8081/index.html. You should now see the contents of the index.html page in your web browser. If not, spend some time troubleshooting before continuing.
Type "close," or click the OSGi stop button on OSGi console view, when you are done.
Adding a Servlet
Start by adding javax.servlet to your dependencies.- Go to the Manifests Wizard (double click on META-INF/MANIFEST.MF if it is not open) and select the dependencies tab.
- Click Add button under Automated Management of Dependencies and choose javax.servlet from the list of bundles.
- Click the Add button next to Imported Packages and add the javax.servlet and javax.servlet.http JARs.
- Click the right mouse and choose Save (or do CNTL-S).
- Create a new class named MyServlet in the com.myfirst.osgi package that extends javax.servlet.http.HttpServlet.
- Add your own Servlet code. The code I used is below.
package com.myfirst.osgi;
import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
public class MyServlet extends HttpServlet {
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
resp.setContentType("text/html");
PrintWriter pw = new PrintWriter (resp.getOutputStream());
pw.println("<html>");
pw.println("<head><title>Servlet Demo</title></head>");
pw.println("<body>");
pw.println("<h2>This servlet is running in an embedded Servlet container on OSGi.</h2>");
pw.println("</body></html>");
pw.close();
}
}
- We will again use an extension to register the Servlet. Return to the Extension tab, click the Add button and set the filter to "
org.eclipse.equinox.http.registry.servlets." Remember to uncheck the check box. - Enter the name of your Servlet class and give it an alias and save your changes.
- The plugin.xml file should now appear as follows:
<plugin>
<extension point="org.eclipse.equinox.http.registry.resources">
<resource
alias="/"
base-name="/WebContent"/>
</extension>
<extension point="org.eclipse.equinox.http.registry.servlets">
<servlet
alias="/MyServlet"
class="com.myfirst.osgi.MyServlet"/>
</extension>
</plugin>
- Select Run --> Open Run Dialog...
- You configuration already includes all the bundles you need, so just click Run.
- OSGi should start normal, as before.
- Re-test your static page by typing http://localhost:8081/index.html
- Test your new Servlet by typing http://localhost:8081/MyServlet
Summary
In this post you extended the project created in my last post to be a Web application bundle. You added a folder for static content and registered the folder's location with Equinox. Next, you cloned your simple Equinox execution environment and added the necessary bundles to run the Jetty embedded application server. This allowed you to accessed some static content from a browser. Finally, you added a Servlet, registered it with Equinox by adding an extension point, and accessed it from a browser.In my next post I will discuss expanding the Equinox execution environment yet again in order to run JSPs and Struts.
Monday, February 4, 2008
Open Your Eyes to OSGi
Open Your Eyes to OSGi
OSGi was not on my radar for 2008. I just happened to start a project this month for a new client that required OSGi. The project involves writing Servlets to run on an Arcom Zeus XScale PSA270 Single Board Computer.
It was only after I started learning about OSGi that I discovered just how popular it has become. Peter Kriens attested to the growing prominence of OSGi in his presentation, OSGi, The Future of Java? , at JavaPolis '07. Here are some other signs that OSGi is on the rise:
- Carlos Perez Blog: Top-5 Java Technologies to Learn in 2008
- Eclipse has been built on OSGi since release 3.
- WebSphere Application Server 6.1 is built on OSGi.
- The JOnAS 5.0.1 application server is built on OSGi.
- First milestone of Spring-OSGi released.
- BEA announced its new Micro-server Architecture (MSA) which is based on OSGi.
- Struts2 is experimenting with hot-pluggable OSGi components.
- The Nuxeo open source ECM platform was ported to Java and OSGi (then bought by JBoss).
Although my immediate need is to develop a CDC-1.0/Foundation-1.0 compliant OSGi package to deploy on an embedded Linux device, my eye is on the longer term goal of building hot-deployable Web applications using Struts 2.0 and Spring-OSGi.
For today's post I am setting up Eclipse 3.3.1 for OSGi on my family's Vista machine. My previous set-up was done with Eclipse 3.2.2 (per client's request) on my Ubuntu workstation--my primary development platform for Byteworks. If you use Eclipse 3.2.2 you will need to install the Service Activator Toolkit (SAT) separately. It magically appeared on Eclipse Europa after installing the Equinox SDK (below).
Get your First Taste of OSGi
Prerequisites: Java must be installed and in your System path.You have already had a taste of OSGi... every time you install an Eclipse plug-in. This is a taste of running OSGi outside of Eclipse.
- Download org.eclipse.osgi_3.3.1.R33x_v20070828.jar from the Eclipse Equinox site.
- Place the JAR in a temporary folder.
- Open a command window and "cd" to the folder where you placed the JAR file.
- Type: java -jar org.eclipse.osgi_3.3.1.R33x_v20070828.jar -console
- The OSGi command prompt should appear ("osgi>").
- To see a short status type "ss" and press ENTER.
C:downloads>java -jar org.eclipse.osgi_3.3.1.R33x_v20070828.jar -console
osgi> ss
Framework is launched.
id State Bundle
0 ACTIVE org.eclipse.osgi_3.3.1.R33x_v20070828
The status output shows that your OSGi environment is empty except for the OSGi bundle itself, which is ACTIVE.
Lets' deploy something else to make this example more interesting. We'll use the install command to download and install the Oscar Bundle Repository plug-in.
- At the osgi> prompt enter this command: install http://oscar-osgi.sf.net/repo/log/log.jar
- Type "ss" to see its status and bundle number.
- Type "start n" where "n" is the bundle number.
- Type "ss" again to see the new status.
Lastly, type "help" to get a list of OSGi commands. We will use "close" to close and exit.
Setting up Eclipse for OSGi Development
Prerequisites - To start this set-up you need Eclipse Europa installed and to be familiar with its use. You should also have a good working knowledge of Java. The desired Java SDK environments also need to be installed. For my CDC-1.0/Foundation-1.0 project I must be able to run on Java1.3.1, so have it installed along with Java 1.4.2, and Java 1.5.Install the Eclipse Equinox SDK Plug-in (optional)
If you are using Eclipse Europa you already have the OSGi tools necessary to complete this example and can skip to OSGi Housekeeping.- Download the latest Eclipse Equinox release. I chose eclipse-equinox-SDK-3.3.1.1.zip.
- Unzip the file and copy the contents of the plugin and features folders to the corresponding folders of your Eclipse installation folder.
OSGi Housekeeping
Make sure that you have installed the JVMs you wish to use and that you have mapped each OSGi Execution Environment to one of the JVMs. Go to Window --> Preferences --> Installed JREs --> Execution Environments. Take a moment to map all the environments listed. You shouldn't have to use Java 1.3 and 1.4. I was just trying to get is close as possible to the target execution environment for my target embedded Linux device.
Creating an OSGi Project
- Choose File --> New --> Project... --> Plug-in Development --> Plug-in Project.
- Give your project a name and choose standard OSGi framework at the bottom of the Project Wizard and click Next.
- Deselect Generate an Activator on the next page. We will do ours by hand later.
- Accept the prompt to switch to the plug-in development perspective.
- Right click on the project and choose Preferences --> Java Build Path --> Libraries.
- Select JRE System Library and click the Edit button.
- Change the JRE System Library to be CDC-1.0/Foundation-1.0.
- Click Java Compiler in the list on the left.
- Change the Compiler Compliance level as needed. In my case, I chose 1.3.
- Click OK to close the Properties dialog.
- When the project opened the Manifests Wizard automatically appeared.
- Open the Dependencies tab and expand the Automated Management of Dependencies section.
- Click the Add button in that section.
- Select the org.eclipse.osgi bundle from the list.
- Click the Add button on the top, right of the screen to import a JAR file from the org.eclipse.osgi bundle.
- Locate org.osgi.framework from the list and select it. The org.osgi.framework JAR contains the Bundle Activator class needed for the next step.
- Select the src folder and create a new package named com.myfirst.osgi.
- Select the com.myfirst.osgi package and create a new class named MyFirstActivator that implements the org.osgi.framework.BundleActivator interface. Check the option to Implement Abstract Methods. The class should look something like this:
- Now we need to add some output to show on the console when it runs. Change the code as shown below.
What did you just do? In order to be able to execute your OSGi bundle stand-alone in the OSGi environment it must have a BundleActivator class that OSGi can call when it starts and stops your bundle. You must now tell OSGi about your bundle's BundleActivator class.
- Click on the Overview tab of the Manifests Wizard.
- Paste "com.myfirst.osgi.MyFirstActivator" in the Activator field.
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: MyFirstOSGiProject Plug-in
Bundle-SymbolicName: MyFirstOSGiProject
Bundle-Version: 1.0.0
Bundle-Activator: com.myfirst.osgi.MyFirstActivator
Import-Package: org.osgi.framework;version="1.4.0"
You are now ready to run your bundle.
Running your OSGi Project
- Right click on your project and choose Run As --> OSGi Framework. A new OSGi Framework Configuration will be created and the OSGi console will open.
- Type "ss" on the OSGi console and you will see a list of over 500 plug-ins. We do not need all of those!
- Click the red X on the OSGi console to stop it (or type close). We want something much simpler.
- Right click again on your project and choose Run As --> Open Run Dialog.
- Click the New Launch Configuration icon in the upper left corner.
- A "New_configuration" will appear under OSGi Framework.
- Find the Name field in the top, center of the screen and change it to "Simple OSGi Configuration."
- Click the Deselect All button to deselect all the OSGi packages in the list.
- Re-select your project package and scroll down to the org.eclipse.osgi package and select it too.
- Uncheck the two check boxes below the list.
- Click the Apply button.
Summary
In this post you learned how to run OSGi from a command prompt as well as how to run it from within Eclipse. You also learned how to make a custom OSGi Execution Environment and pick and choose which bundles to activate. Finally, you learned about how to implement the BundleActivator interface and configure your project such that the OSGI runtime will call your BundleActivator.In my next post we will turn our attention to deploying and running Servlets in OSGi bundles.
Steven Mitchell
Sunday, January 27, 2008
Making it with Maven2 and Eclipse
In my post "Retooling the Site" one of my objectives was to switch to Maven2. This weekend I am starting a project for a new client, and I have set out to do the following: 1) Create instructions for my client (also named, Steve) to get him jump started with Maven2; 2) Create a Maven2 project shell for my client with JUnit Report, CheckStyle reports, JavaDoc, and FindBugs; 3) Convert ByteworksInc.com to Maven2; and 4) Convert HomesAssn.com to Maven2. Welcome to my blog, Steve. Feel free to stop after section "Tell Eclipse About the Local Repository".
Prerequisites
- The Java 5 SDK is installed.
- The JAVA_HOME System variable is set.
- The PATH System variable includes %JAVA_HOME%/bin (Windows) or $JAVA_HOME/bin (Linux).
- Eclipse is installed (I prefer the Eclipse Web Tools all-in-one bundle since I do web development).
If you are using OS X you'll need to create .bash_profile and add something like /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home as your JAVA_HOME.
Installing Maven2
For those of you new to Maven2, there is some set-up to do, as there was with ANT, with the additional concept of local Maven2 repository.
- Download the latest Maven2 distribution from http://maven.apache.org/download.html.
- Unzip the distribution to a directory of your choosing, such as "C:\Program Files\Apache Software Foundation".
- Add the Maven2 installation directory as an environment variable by opening up the system properties (WinKey + Pause), selecting the "Advanced" tab, and the "Environment Variables" button, then adding M2_HOME with the path that you chose (e.g. "C:Program FilesApache Software Foundationapache-maven-2.0.8" ).
- Add the Maven2 bin directory to your path by opening up the system properties (WinKey + Pause), selecting the "Advanced" tab, and the "Environment Variables" button, then editing the PATH variable and adding ";%M2_HOME%\bin".
- Open a command window and type "mvn -version" to verify that Maven is installed correctly.
Create You Local Repository
Maven2 will automatically create your local repository, but it will not creating the settings.xml file automatically. You need settings.xml for the Maven Eclipse plug-in, and it is also the file in which you configure your proxy settings if you are behind a web proxy. Here we manually configure the local repository so that the Maven2 Eclipse plug-in finds everything it is looking for and you learn a little about it.
- cd %USERPROFILE%
- mkdir .m2\repository
- Add your local Maven2 repository directory a user local environment variable by opening up the system properties (WinKey + Pause), selecting the "Advanced" tab, and the "Environment Variables" button, then adding M2_REPO with the path "%HOMEPATH%\.m2\repository". Note: be sure you create a local user variable because %HOMEPATH% cannot be used for a System variable.
- Create a file in %HOMEPATH%\.m2 named settings.xml.
- Add the following content.
For OS X your .bash_profile will look something like this:
Installing the Maven2 Plug-in in Eclipse
The Maven2 plug-in can be installed using the Eclipse Update feature:
- Launch Eclipse.
- Go to Help --> Software Updates --> Find and Install.
- Select Search for new features to install, and then click next.
- Click the New Remote Site button and set the name to "M2Eclipse" and set the URL to http://m2eclipse.sonatype.org/update/.
- "M2Eclipse" should now appear in the Sites to include in search list and should already be checked. Click Finish.
- When a dialog to select the features to install appears select the "Maven Integration for Eclipse" feature and then click Next.
- Accept the license agreement then click Next.
- Click Finish on the next dialog page and then click Install All to complete the installation.
Tell Eclipse About the Local Repository
- Use this command to tell Eclipse about your local repository: "mvn -Declipse.workspace=<path-to-eclipse-workspace> eclipse:add-maven-repo"
- The command will download a variety of dependencies.
- Change to %M2_REPO% to see that it now contains some JAR files.
Create a Simple Java Project
To create a simple Java project, perhaps for a batch program, issue the following commands:
- Change to your workspace or development directory.
- Run "mvn archetype:create -DgroupId=com.mysite.myproject -DartifactId=MyProjectName". This will create a directory named "MyProjectName" containing a pom.xml and the Maven2 file structure.
- Type "cd MyProjectName" to change to the newly created directory.
- Run "mvn compile" and "mvn test" to test the project you just created.
- Run "mvn eclipse:eclipse" to create the .project and .classpath files for Eclipse.
- Launch Eclipse and select File --> Import --> General --> Existing project and navigate to the newly created Maven2 project.
- When project appears in your workspace, right-click and choose Maven --> Enable Dependency Management.
- Right-click on the project again and choose Maven --> Update Source Folders.
- Open the src/test/java folder and run AppTest.java as a JUnit test.
Create a Web Application
Follow the same steps as above, but run this command in step 2: "mvn archetype:create -DgroupId=com.mysite.myproject -DartifactId=my-webapp -DarchetypeArtifactId=maven-archetype-webapp". Change to the "my-webapp" directory and run "mvn package" to create a WAR file in the "target" directory.
Create a Multiple Module Project
Okay - here is where I lose part of my ANT audience. Anything goes in ANT: you can generate all the artifacts you wish. Maven is more restrictive. Each pom.xml file can have only one target; therefore, if you need a JAR and WAR, then you need two modules: a core module to generate the JAR file, and a site module, which depends on the core module, to generate the WAR. Both modules are nested under a parent project module. This is also where I lose some Eclipse readers since Eclipse doesn't allow nested modules. Eclipse users must load each child module as separate projects, as opposed to JetBrains' Intellij IDEA, which supports nested projects natively. With that being said, here is what we will do to set-up a multi-module Maven2 project in Eclipse:
Create the Parent pom.xml file:
- Change to your workspace or development directory.
- Run "mvn archetype:create -DgroupId=com.mycompany.myproject -DartifactId=MyMultiModuleProject"
- Type "cd MyMultiModuleProject" to change to the new parent directory.
- Run "rmdir /S src" to remove the src directory from the parent directory ("rm -Rf src" on OS X or Linux).
- Edit the pom.xml file and change "<packaging>jar</packaging>" to "<packaging>pom</packaging>".
- Run "mvn install" to install the MyMultiModuleProject-1.0-SNAPSHOT pom in .m2/repository.
This is where you see the difference between Eclipse and Intellij Idea. If you run "mvn idea:idea" for a pom project, Maven creates the Idea project files; however, if you run "mvn eclipse:eclipse" for a pom project before creating the child projects nothing happens. This behavior is intentional to avoid confusion over Eclipse not supporting hierarchical projects.
Create the child modules and configure the parent pom.xml:
- Change to the MyMultiModuleProject directory.
- Run "mvn archetype:create -DgroupId=com.mycompany.myproject.core -DartifactId=myproject-core" to create the core JAR module.
- Run "mvn archetype:create -DgroupId=com.mycompany.myproject -DartifactId=myproject-site -DarchetypeArtifactId=maven-archetype-webapp" to create the site WAR module.
- View the MyMultiModuleProject/pom.xml and note that it now contains the following referece to the child modules:
Steps 2 and 3 above also inserted references in the child pom files. View the contents of MyMultiModuleProject/myproject-core/pom.xml and MyMultiModuleProject/myproject-site/pom.xml and note that they both contain the following reference to the parent pom:
Add a Dependency to the Site pom for the Core pom:
- Add the following dependency to MyMultiModuleProject/myproject-site/pom.xml:
Build and install all modules in the project:
- Change back to the MyMultiModuleProject parent directory.
- Re-run "mvn install". The <module> tags will cause the core and site modules to be installed.
- Re-run "mvn eclipse:eclipse" in the parent directory to generate the .project and .classpath files in the both the core and site modules.
Load the projects into Eclipse:
- Launch Eclipse and choose File --> New Project.
- Give the Project a name an click "Create Project from existing source."
- Browse to the parent project directory and click Choose.
- Click Finish. The project will appear in the workspace with errors.
- Right click on the parent project and select Maven --> Enable Maven Dependencies.
- Right click on the parent project and select Maven --> Enable nested modules.
That was a lot to cover in one blog post, but it isn't that complicated once you are familiar with the steps I hope you found this information useful. Try the resources below to learn more.
Resources:
Maven2 Quick Reference Card
Installing Maven2 Plug-in for Eclipse
Guide to Using Eclipse with Maven2
Reference for Maven2 settings.xml file
Maven2 Archetype Plug-in
Steven Mitchell
Subscribe to:
Posts (Atom)
