Saturday, 21 February 2015

RPC, Protobuf, Scala

Seen too many custom frameworks for RPC, asynchronous callbacks, custom serialization, etc...... There are great frameworks out there to handle this. Let's have a look at a simple Search service, asynchronous request, using protobuf and Scala. Some *.proto def where the services are defined, and the data to marshall in and out.
syntax = "proto2";

package myprotocol;

message SearchRequest {
 required int32 requestid = 1;
}

message SearchResponse {
 required int32 requestid = 1;
 required int32 responseid = 2;
}
A simple RPC call definition:
syntax = "proto2";
option java_generic_services = true;

package myprotocol;

import "Search.proto";

service SearchService {
  rpc Search (SearchRequest) returns (SearchResponse);
}
Then you define two classes...the one that implements your RpcChannel is the one where you plugin your underlying messaging system, such as JMS, Solace, Tibco RV, or sockets...
package demo

import java.util.logging.Logger

import com.google.protobuf.Descriptors.MethodDescriptor
import com.google.protobuf.Message
import com.google.protobuf.RpcCallback
import com.google.protobuf.RpcChannel
import com.google.protobuf.RpcController

import myprotocol.Search.SearchRequest
import myprotocol.Search.SearchResponse

class SChannel extends RpcChannel {
  val logger = Logger.getLogger("SChannel")
  
  override def callMethod(methodDescriptor: MethodDescriptor , 
                          rpcController: RpcController , 
                          m1: Message, 
                          m2: Message, 
                          rpcCallback: RpcCallback[Message] ) {
    m1 match {
      case sr: SearchRequest => {
        val response = SearchResponse.newBuilder.setRequestid(sr.getRequestid).setResponseid(2).build
        rpcCallback.run(response)
      }
      case _ => logger.warning(s"Not handling message type $m1 yet")
    }
  } //  override def callMethod
}

class SController extends RpcController {
  val logger = Logger.getLogger("SController")
  
  override def reset() {
    logger.info("reset()")
  }

  override def failed(): Boolean = {
    logger.info("failed()")
    false
  }

  override def errorText(): String = {
    logger.info("errorText()")
    null
  }

  override def startCancel() {
    logger.info("startCancel()")
  }

  override def setFailed(s: String) {
    logger.info(s"setFailed($s")
  }

  override def isCanceled():Boolean = {
    logger.info("isCanceled()")
    false
  }

  override def notifyOnCancel(rpccallback: RpcCallback[Object]) {
    logger.info(s"notifyOnCancel($rpccallback)")
  }
}
Edited I Forgot the code to actually use those classes :-)
package demo

import java.util.logging.Logger

import com.google.protobuf.RpcCallback

import myprotocol.Search.SearchRequest
import myprotocol.Search.SearchResponse
import myprotocol.Services

object RPCDemo {
  val logger = Logger.getLogger("RPCDemo")
  
  def main(args: Array[String]): Unit = {
    val channel = new SChannel
    val controller = new SController
    val services = Services.SearchService.newStub(channel)
    val request = SearchRequest.newBuilder().setRequestid(1).build
    val callback = new RpcCallback[SearchResponse] {
      override def run(obj: SearchResponse) = logger.info(s"Received on RpcCallback $obj")
    }
    services.search(controller, request, callback)
    
    logger.info("Sleeping 5 seconds")
    Thread.sleep(5000)
    logger.info("Ciao")
  }
}
For example, if you are looking for a pre-built TCP/IP socket implementation on top of protobuf, check out protobuf-socket-rpc A detailed view of a custom channel can be seen there: RpcChannelImpl.java

Edited II You can obviously define an implict instead of this callback a la Java:
  import scala.language.implicitConversions
  implicit def f2cb(f: (SearchResponse) => Unit) = new RpcCallback[SearchResponse] {
    override def run(sr: SearchResponse) = f(sr)
  }

  services.search(controller, request, (resp: SearchResponse) => {
    logger.info(s"Received on RpcCallback $resp")
  })
... Voila.

Sunday, 11 January 2015

Ouray! Just completed my Machine Learning Course!

Completed my ML course from Coursera! Lots of work!

RedMart is hiring

If you are interested, please check this link. Another neat implementation in Scala is fairly easy...

Monday, 15 December 2014

So much fun recently

Had lots of fun recently: played with Scalatra, RESTful APIs, Slick, all the web services security aspects like HMAC, etc... Really fun. I am quite amazed that I never had to work on web services until now... but hey, there is always a first time.
Special Kudos to Scalatra -- just love this framework, although the integration with Swagger can be a real pain.

Tuesday, 9 December 2014

Sunday, 12 October 2014

Private Key for Digital Signature, KeyStore, PKCS#12

The following demonstrates the use of the Java security API to digitally sign a document.
Before we get to the Scala code, we must first use Java's keytool to create a self-signed certificate.
In real-life, you'd probably buy one from Verisign for example. The archive file format chosen is PKCS12.
keytool -genkey
        -alias tj
        -keystore mykeystore
        -storepass storepass
        -validity 365
        -keyalg RSA
        -keysize 2048
        -storetype pkcs12
Once you have answered the questions, you can list the aliases using the following command:
keytool -keystore mykeystore 
        -storepass storepass 
        -list 
        -storetype pkcs12
Should output:
Keystore type: PKCS12
Keystore provider: SunJSSE

Your keystore contains 1 entry

tj, Oct 12, 2014, PrivateKeyEntry,
Certificate fingerprint (SHA1): 52:6B:0D:05:9E:CE:5A:CA:5E:EF:74:C9:51:FE:46:8D:E6:CE:4F:11
Let's get to the coding part. We will extract the private key from the store to digitally sign a document. I usually first create a digest from the initial document and store this digest as base 64. But there is no need for this step.
  def createDigest(legalDocument: String): Array[Byte] = {
    val md = MessageDigest.getInstance("SHA-256")
    md.update(legalDocument.getBytes)
    md.digest()
  }
to be called like this:
val alias = "tj"
val password = "storepass".toCharArray
val legalDoc = createDigest("This is a legal document I must digitally sign")
legalDoc is an array of bytes - it contains a SHA-256 digest from the initial document.

The code to list the keystore entries, just to make sure we are on the right path:
  def listStoreEntries(password: Array[Char]): KeyStore = {
    val keyStoreDefaultType = KeyStore.getDefaultType
    val keyStore = KeyStore.getInstance("pkcs12")
    keyStore.load(new FileInputStream("mykeystore"), password)

    val aliases = keyStore.aliases()
    while(aliases.hasMoreElements) {
      val alias = aliases.nextElement()
      log(s" Alias: $alias")
    }

    keyStore
  }
listStoreEntries returns the initialized keystore. Let's now sign the legal document and return a base 64 encoded digital signature:
  def signLegalDocument(keystore: KeyStore, alias: String, password: Array[Char], legalDoc: Array[Byte]): String = {
    val privateKey = keystore.getKey(alias, password)
    val dsig = Signature.getInstance("MD5withRSA")
    dsig.initSign(privateKey.asInstanceOf[PrivateKey])
    dsig.update(legalDoc)
    val signature = dsig.sign()
    Base64.getEncoder.encodeToString(signature)
  }
And the code to verify the signature:
  def verifyDigitalSignature(keystore: KeyStore, alias: String, legalDoc: Array[Byte], signature: String): Unit = {
    val certificate = keystore.getCertificate(alias)
    val x509Certificate = certificate.asInstanceOf[X509Certificate]
    val publicKey = x509Certificate.getPublicKey
    val dsig = Signature.getInstance("MD5withRSA")
    dsig.initVerify(publicKey)

    val sig = Base64.getDecoder.decode(signature)
    dsig.update(legalDoc)
    val verifiedSig = dsig.verify(sig)
    log(s"Has the legal document signature successfully been verified? $verifiedSig")
    require(verifiedSig == true)
  }
The whole main method:
  def main(args: Array[String]) {
    val alias = "tj"
    val password = "storepass".toCharArray
    val legalDoc = createDigest("This is a legal document I must digitally sign")
    val keyStore = listStoreEntries(password)

    val signature = signLegalDocument(keyStore, alias, password, legalDoc)
    verifyDigitalSignature(keyStore, alias, legalDoc, signature)
  }

  def log(ref: Any) = println(ref)
How do I import my self-signed certificate into Windows cert manager?
Just rename the keystore from mykeystore to mykeystore.pfx On Windows, simple double-click on it and follow the instructions.
Open certmgr.msc via the run command or from a DOS console - under Certificates - Currrent User, Personal, Certificates you should see your previously create certificate using Java's keytool.

Friday, 5 September 2014

Scalatra

Just found this neat Scala project, Scalatra.
Quite amazing I did not know it... but looks pretty cool: I love the integration with Slick and Akka...

Blog Archive