Monday 21 November 2011

Openldap2.4.26, BerkeleyDB-5.2 and Sasl from scratch

In these days I've deployed a openldap server with openldap-2.4.26, BerkeleyDB.5.2 and cyrus-sasl-2.1.23.

First of all make sure no *-dev packeges are installed for bdb or for sasl.

Start with BerkeleyDB: download from the oracle site, expand and:
cd db-5.2.36/
cd build_unix && ../dist/configure --prefix=/usr/local/BerkeleyDB.5.2/
make
sudo make install

Now it's time for sasl. I got the cyrus-sasl-2.1.23.tar.gz package from: http://cyrusimap.web.cmu.edu/mediawiki/index.php/Cyrus_SASL

Expand it and apply patch: http://repos.archlinuxppc.org/wsvn/filedetails.php?repname=packages&path=%2Fcyrus-sasl-plugins%2Ftrunk%2Fcyrus-sasl-2.1.23-db5-fix.patch.

The patch is actually a patch to two files. To apply a patch a
patch -p1 < patchfile
should work. If there are rejected chunks, move to the dir of the file to be modified and apply only the part of the patch that refers to that file. Write a compile script:
#!/bin/bash

export LDFLAGS="-L/usr/local/BerkeleyDB.5.2/lib"
export LD_LIBRARY_PATH=/usr/local/BerkeleyDB.5.2/lib/

sed -i.bak 's/#elif WITH_DES/#elif defined(WITH_DES)/' \
    plugins/digestmd5.c &&
env CPPFLAGS="-I/usr/local/BerkeleyDB.5.2/include/" \
LDFLAGS="-L/usr/local/BerkeleyDB.5.2/lib/ -R/usr/local/BerkeleyDB.5.2/lib" \
./configure --with-dblib=berkeley --prefix=/usr/local/sasl-2.1.23/
Execute the compile script followed by make and sudo make install. Remember to create the soft link from /usr/local/sasl-2.1.23/lib/sasl2 to /usr/lib/sasl2.
sudo ln -s /usr/local/sasl-2.1.23/lib/sasl2 .
To compile openldap, download and expand the packege, write a compilation script:
#!/bin/bash

export LDFLAGS="-L/usr/local/BerkeleyDB.5.2/lib"
export LD_LIBRARY_PATH=/usr/local/BerkeleyDB.5.2/lib/

env CPPFLAGS="-I/usr/local/BerkeleyDB.5.2/include/ \
 -I/usr/local/sasl-2.1.23/include/" \
LDFLAGS="-L/usr/local/BerkeleyDB.5.2/lib/ -L/usr/local/sasl-2.1.23/lib/" \
./configure --enable-crypt --enable-hdb=yes --enable-ppolicy \
--with-cyrus-sasl --with-tls=openssl --enable-overlays \
--enable-valsort --localstatedir=/var/lib/ --prefix=/usr/local/openldap-2.4.26

Then run the command, make depend, make and sudo make install.

Thursday 22 September 2011

jruby processor with activemq-5.5.0

Activemq is already in bundle with camel. The camel config file is in $ACTIVEMQ_HOME/conf/camel.xml.

But you need something more to use a jruby processor with activemq:
- add lang namespace to camel.xml;
- add a aopalliance.jar and jruby.jar to classpath (and more, if you need logback);
- write and register jruby bean.

Add namespace to camel.xml: just replace header with:
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:lang="http://www.springframework.org/schema/lang"
       xmlns:p="http://www.springframework.org/schema/p"
       xsi:schemaLocation="
                           http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
                           http://www.springframework.org/schema/lang
                           http://www.springframework.org/schema/lang/spring-lang-2.5.xsd
                           http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">

Add a aopalliance.jar to classpath: download springsource-2.5 with all dependencies. Unzip it and get aopalliance.jar from the lib/aopalliance directory.

Activemq $CLASSPATH is defined as $ACTIVEMQ_HOME/conf, so create a ruby folder in conf and add a jruby bean named ruby_processor.rb:
java_import "org.apache.camel.Exchange"
java_import "org.apache.camel.Processor"
java_import "org.slf4j.Logger"
java_import "org.slf4j.LoggerFactory"
java_import "ch.qos.logback.classic.LoggerContext"

java_package "com.test.camel"

class RubyProcessor
  include Processor

  def initialize
    @logger = LoggerFactory.getLogger("com.test.camel.#{File.basename(__FILE__, ".rb")}");
  end  

  java_signature "void process(Exchange exchange)"
  def process(exchange)
    test = exchange.in.headers['test']
    @logger.info("RubyProcessor: header test is: #{test}")
    exchange.in.set_header("Issuer", "rubyProcessor")
  end
end
Please note this bean does very few: just dump the value of a header to a log file, then sets a new header. It is just e proof-of-concept bean.

In camel.xml add a test route like:
<from uri="activemq:example.A"/>
     <bean ref="rubyProcessor" />
            <to uri="activemq:example.B" />

 <lang:jruby id="rubyProcessor"
              script-interfaces="org.apache.camel.Processor"
              script-source="classpath:ruby/ruby_processor.rb">
  </lang:jruby>

The ticky part.


To have logback work first you need to include in lib folder logback-classic and logback-core then I had to upgrade slf4j to 1.6.2 which means copying a couple of jars both in lib and in lib/optional.

The logback.xml can be created in conf, logfile from logback are written in $ACTIVEMQ_HOME.

Than I noticed the ruby scripting fails to work in the routes. I had to replace the <ruby> tags with <simple>.

Wednesday 14 September 2011

Send a mail with ruby1.8, ruby1.9 and jruby

I lost the best part of this morning to send a mail with ruby with the constrain it should work with ruby1.8, ruby1.9 and jruby.

This is my best effort:
#!/usr/bin/env ruby
#ENCODING: UTF-8

require 'rubygems'
require 'net/smtp'
require 'mail'

TO = 'you@test.com'
FROM = 'me@test.com'

mail = Mail.new do
  from    FROM
  to      TO
  subject 'This is a test email'
  body    "Test\n\nNow is the time #{Time.now}\n"
end

Net::SMTP.start('localhost') do |smtp|
  smtp.send_message(mail.to_s, TO, FROM)
end

I hope it could save someone's time!

Tuesday 13 September 2011

Monitor activemq queue-size with jruby jmx

The aim is to know when a certain threshold of queued messages is reached on certain destinations. For example, if new messages show up in DLQ (dead letter queue) it might be a hint of some problems.

First of all enable jmx for activemq on a free port (11099, for example). For activemq-5.5.0 modify /etc/default/activemq as follows:
ACTIVEMQ_SUNJMX_START="-Dcom.sun.management.jmxremote.port=11099 
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.authenticate=false"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.ssl=false"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote"

Restart activemq and confirm something is listening on port 11099:
$ sudo lsof -i :11099
COMMAND   PID     USER   FD   TYPE    DEVICE SIZE NODE NAME
java    23206 activemq   10u  IPv6 110495619       TCP *:11099 (LISTEN)

Install jmx gem:
sudo jruby -S gem install jmx

Then a simple script like this can query the size of the queue ActiveMQ.DLQ for further actions:
require 'java'
require 'rubygems'
require 'jmx'

client = JMX.connect(:port => 11099)

queue = client["org.apache.activemq:BrokerName=bacedifo,Type=Queue,Destination=ActiveMQ.DLQ"]
#puts queue.attributes

puts queue["QueueSize"]

Friday 22 July 2011

Camel 2.7.0 and SLF4J

While upgrading an application written from camel2.6.0 to 2.7.0, I hit the following error:
SLF4J: The requested version 1.6 by your slf4j binding is not compatible with \
 [1.5.5, 1.5.6, 1.5.7, 1.5.8, 1.5.9, 1.5.10, 1.5.11]
which shortly prevented camel from starting as soon as a bean tries to log.

The solution looks quite easy: just add to pom.xml:
<properties>
    <slf4j-version>1.6.1</slf4j-version>
</properties>
and in dependencies:
 <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-api</artifactId>
      <version>${slf4j-version}</version>
    </dependency>

I also deleted the log4j dependency.

I'm not really sure whether thir behaviour is somehow caused by logback, which I used also before, in camel2.6.0.

Thursday 30 June 2011

Camel producerTemplate in ruby

The aim is to send messagges to a queue with the camel producerTemplate. producerTemplate can be invoked from a Processor. This time I'm going to use a jruby processor.

First of all generate a camel project with:
mvn archetype:generate \\
-DarchetypeGroupId=org.apache.camel.archetypes \\ 
-DarchetypeArtifactId=camel-archetype-java \\
-DarchetypeVersion=2.7.0 -DgroupId=com.test -DartifactId=producer

Then modify pom.xml to import all dependencied:
<name>## Producer ##</name>
  <url>http://www.myorganization.org</url>
  <properties>
    <camel-version>2.7.0</camel-version>
    <log4j-version>1.2.16</log4j-version>
    <logback-version>0.9.24</logback-version>
    <activemq-version>5.5.0</activemq-version>
    <jruby-version>1.6.2</jruby-version> 
    <org.springframework.version>3.0.5.RELEASE</org.springframework.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.apache.camel</groupId>
      <artifactId>camel-core</artifactId>
      <version>${camel-version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.camel</groupId>
      <artifactId>camel-spring</artifactId>
      <version>${camel-version}</version>
    </dependency>
    
    <!-- logging -->
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-log4j12</artifactId>
      <version>1.5.11</version>
    </dependency>
    <dependency>
      <groupId>log4j</groupId> 
      <artifactId>log4j</artifactId> 
      <version>1.2.16</version> 
    </dependency>

    <dependency>
      <groupId>ch.qos.logback</groupId>
      <artifactId>logback-core</artifactId>
      <version>${logback-version}</version>
    </dependency>
    <dependency>
      <groupId>ch.qos.logback</groupId>
      <artifactId>logback-classic</artifactId>
      <version>${logback-version}</version>
    </dependency>
    <dependency>
      <groupId>org.jruby</groupId>
      <artifactId>jruby-complete</artifactId>
      <version>${jruby-version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.camel</groupId>
      <artifactId>camel-jms</artifactId>
      <version>${camel-version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.activemq</groupId>
      <artifactId>activemq-camel</artifactId>
      <version>${activemq-version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.activemq</groupId>
      <artifactId>activemq-core</artifactId>
      <version>${activemq-version}</version>
    </dependency>
    <dependency>
      <groupId>org.apache.camel</groupId>
      <artifactId>camel-script</artifactId>
      <version>${camel-version}</version>
    </dependency>
    <dependency>
      <groupId>cglib</groupId>
      <artifactId>cglib-nodep</artifactId>
      <version>2.1_3</version>
    </dependency>
  </dependencies>

Define the resources needed in file src/main/resources/META-INF/spring/camel-context.xml:
<?xml version="1.0" encoding="UTF-8"?>
<!--
    Licensed to the Apache Software Foundation (ASF) under one or more
    contributor license agreements.  See the NOTICE file distributed with
    this work for additional information regarding copyright ownership.
    The ASF licenses this file to You under the Apache License, Version 2.0
    (the "License"); you may not use this file except in compliance with
    the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.
-->

<!-- Configures the Camel Context-->

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:lang="http://www.springframework.org/schema/lang"
       xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd 
      http://www.springframework.org/schema/lang 
      http://www.springframework.org/schema/lang/spring-lang-2.5.xsd
      http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
  

 <bean id="credentials" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name='location'>
      <value>credential.properties</value>
    </property>
  </bean>

<!--
<bean id="credentials" class="org.apache.camel.component.properties.PropertiesComponent">
  <property name="location" value="classpath:credential.properties" />
</bean>
-->
  <bean id="local-activemq" class="org.apache.activemq.camel.component.ActiveMQComponent" >
    <property name="connectionFactory">
      <bean class="org.apache.activemq.ActiveMQConnectionFactory">
        <property name="brokerURL" value="failover://(tcp://localhost:61616)" />
        <property name="userName" value="${activemq.username}"/>
        <property name="password" value="${activemq.password}"/>
      </bean>
    </property>
  </bean>

  <lang:jruby id="producer"
              script-interfaces="org.apache.camel.Processor"
              script-source="classpath:ruby/producer.rb">
    <lang:property name="template" ref="producerTemplate" />
  </lang:jruby>


  <camelContext xmlns="http://camel.apache.org/schema/spring">
    <package>com.test</package>

    <template id="producerTemplate"/>
  </camelContext>

</beans>

Please note the added 'lang' namespace in premable and relative expanded xsi:schemaLocation. Missing that will issue a error.

Note also that credential for activemq should be written in src/main/resources/credential.properties, for example:
activemq.username=system
activemq.password=manager

Than create the route in file src/main/java/com/test/MyRouteBuilder.java:
/**
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package com.test;

import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.spring.Main;

import static org.apache.camel.builder.xml.XPathBuilder.xpath;

/**
 * A Camel Router
 */
public class MyRouteBuilder extends RouteBuilder {

    /**
     * A main() so we can easily run these routing rules in our IDE
     */
    public static void main(String... args) throws Exception {
        Main.main(args);
    }

    /**
     * Lets configure the Camel routing rules using Java code...
     */
    public void configure() {

 from("timer://producer_timer?fixedRate=true&delay=5000&period=10000").
     errorHandler(loggingErrorHandler("com.test")).
     id("producer").
     setBody(constant("Hallo from producer")).
     beanRef("producer");

    }
}
This is a simple route which triggers each 10 seconds, creates a empty message, changes its body to 'Hallo from producer' then sends it to producer bean.



Now it's time to write the processor, which in turn uses producerTemplate. According to camel-context.xml source file should be in src/main/resources/ruby/producer.rb:
require 'java'

java_import "org.apache.camel.Exchange"
java_import "org.apache.camel.Processor"

java_import "org.slf4j.Logger"
java_import "org.slf4j.LoggerFactory"
java_import "ch.qos.logback.classic.LoggerContext"

java_package "it.unimore.cesia"

class Producer
  include Processor

  def initialize
    @logger = LoggerFactory.getLogger("com.test.producer")
    @logger.info("Hello from Producer")
  end  

  def setTemplate(template)
    @@template = template
  end

#java_annotation 'Exchange'
java_signature 'void process(Exchange exchange)'
  def process(exchange)
    msg = exchange.in.body
    @logger.info("Producing at #{Time.now.strftime('%H:%M')}")
    @@template.sendBody("local-activemq:queue:producer.audit", "#{msg} at #{Time.now.strftime('%H:%M')}")

  end

end

Route can be activated with:
mvn camel:run

If you want to activate logging, create a logback.xml file in src/main/resource. For example for a syslog/file logging:
<configuration scan="true" scanPeriod="30 seconds">
  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <File>producer.log</File>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <FileNamePattern>producer-%d{yyyy-MM-dd}.log</FileNamePattern>
      <maxHistory>30</maxHistory>
    </rollingPolicy>
    <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<!--      <pattern>%date %level [%thread] %logger{10} %msg%n</pattern> -->
      <Pattern>%date %msg%n</Pattern>
      <charset>UTF-8</charset>
    </encoder>
  </appender>

  <appender name="SYSLOG"
    class="ch.qos.logback.classic.net.SyslogAppender">
    <SyslogHost>localhost</SyslogHost>
    <Facility>DAEMON</Facility>
    <SuffixPattern>[%logger] %msg</SuffixPattern>
  </appender>

  <logger name="com.test.producer" additivity="false">
    <level value="DEBUG" />
    <appender-ref ref="FILE" /> 
    <appender-ref ref="SYSLOG" />
  </logger>

  <logger name="com.test" additivity="false">
    <level value="INFO"/>
    <appender-ref ref="FILE" /> 
    <appender-ref ref="SYSLOG" />
  </logger>


  <root level="ERROR">
<!--
    <appender-ref ref="FILE" />
    <appender-ref ref="STDOUT" /> -->
    <appender-ref ref="SYSLOG" />
  </root>
</configuration>

Monday 13 June 2011

My first camel processor in scala

First create a maven project with:
mvn archetype:generate -DarchetypeGroupId=org.apache.camel.archetypes \\
-DarchetypeArtifactId=camel-archetype-scala -DarchetypeVersion=2.7.2 \\
-DgroupId=com.unitest -DartifactId=my_test

It creates a my_test project with a route defined in src/main/scala/com/unitest/MyRouteBuilder.scala, which can be modified as:

"timer://foo?fixedRate=true&delay=5s&period=10s" ==> {
     bean("myBean") 
     to("log:bean")
   }

Where myBean is referenced in src/main/resources/META-INF/spring/camel-context.xml as:
<bean id="myBean" class="com.unitest.MyBean"/>

and class MyBean, in folder src/main/scala/com/unitest is:
package com.unitest

import org.apache.camel.Exchange

class MyBean {

  def process(exchange: Exchange) {
    exchange.getIn.setBody("EpiReplicaQuery test")
   }

Execute with mvn camel:run and that's all.

Thursday 9 June 2011

juniper ssl vpn appliance and shibboleth2.2.1

First step is reading:
http://shibboleth.1660669.n2.nabble.com/Juniper-SSLVPN-integration-td3575845.html

There are also some very clear instuction by P. Geenens (pgeneens@juniper.net).

Something I was unaware was the need to create a Sign-in policy, for instance:
User URLs  Sign-In Page     Authentication Realm(s)
*/saml/  my Sign-In Page  shibboleth

Where shibboleth is the label of the Authentication Realm which uses the shibboleth authentication server.

Now, change the value of Source Site Inter-Site Transfer Service URL to
https://omissis.unitest.com/idp/profile/Shibboleth/SSO?providerId=vpn.unitest.com&shire=https://vpn.unitest.com/dana-na/auth/saml-consumer.cgi&target=https://vpn.unitest.com/saml
where /saml is the path of the Sign-in policy page.

Now as the user connects to https://vpn.unitest.com/saml, she is redirected to shibboleth IdP login page and than back to ssl vpn.

Add an alternative to update-alternatives

Let's start from the use case: you install firefox4.0.1 anf just want it starts from the 'Web Browser' button in the lxde menu bar.

It looks it was about configuring the x-www-browser alternative.

I started from update-alternatives --install, but wasn't able to figure out how to name the 4 parameters.

While doing some tries, I deleted the /usr/bin/x-www-browser link, which pointed to /etc/alternatives/x-www-browser which was a link to /usr/bin/iceweasel.

I gave up and linked /usr/local/src/firefox/firefox to /etc/alternatives/x-www-browser and again to /usr/bin/x-www-browser and opla'! Not only the 'Web Browser' button works, but also the alternative stuff is ok:

sudo update-alternatives --list x-www-browser
/usr/bin/epiphany-browser
/usr/bin/iceweasel
/usr/local/src/firefox/firefox