Random Number Generator provider abstract class

I had a need to support multiple hardware random number generators (RNG) in a project. I started with an RngProvider abstract class. This class defines the functionality that will be the same and defines a method that will contain the functionality that is unique per RNG.

Queue size
We set the desired queue length to be 80k integers per hardware RNG which will allow us to cache enough integers to make over 3k decks of cards. 80k integers per integration doesn’t take up any space worth worrying about. 80k * 4 / 1024 = 312.5kb. That is practically free. Cache away. A queue fill operation is going to be triggered when the queue is 1/4 full.

LinkedBlockingQueue
I chose a linked blocking queue because I needed synchronized access to prevent the random integer from being selected more than once. The blocking operations are within the BlockingQueue interface. I used put() and take() to queue and dequeue with blocking until the queue is available.

Available
I created this property as a way to add or remove the device without having to stop/start the server, if the device supports that kind of functionality. The thinking was that the constructor would set it to available, the nextInt() would check to see if it was empty and set it to unavailable, if necessary, and the implementation of the getRandomIntegerData() method would set it to available as it adds integers.

getRandomIntegerData()
This method defines the method that would be filled in by the specific implementation needed by the various hardware devices.

nextInt()
This method is used to retrieve an integer from the specific integer queue and also start the thread that will manage adding integers to the queue, if necessary.

FillQueue
This thread is created when a number is requested to be removed from the queue. If the size of the queue has dropped below a specified number, then the getRandomIntegerData() method is called to replenish the integer data. The block of code that determines if the integer data is to be replenished is synchronized to prevent multiple thread executions from trying to replenish the queue at the same time.

package com.cdi.rng;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;

/**
 * Interface to define random number generator providers.
 * @author norris.shelton
 */
public abstract class RngProvider {
    /** SLF4J logger object. */
    protected Logger logger = LoggerFactory.getLogger(getClass());
    /** The desired size of the integer queue. */
    public static final int DESIRED_QUEUE_LENGTH = 80000;
    /** The minimum integer queue size. */
    public static final int MIN_QUEUE_LENGTH = DESIRED_QUEUE_LENGTH / 4;
    /** Is this RNG provider available. */
    boolean available = false;
    /** Queue of integers for consumption. */
    protected BlockingQueue<Integer> integerQueue = new LinkedBlockingQueue<>(DESIRED_QUEUE_LENGTH);

    /**
     * Is this random number generator provider available.
     * @return true if available for use, else false
     */
    public boolean isAvailable() {
        return available;
    }

    /**
     * Sets that this random number generator is available.
     * @param available true if available, else false
     */
    public void setAvailable(boolean available) {
        this.available = available;
    }

    /**
     * Fills the integer queue with integers.
     * @throws Exception
     */
    public abstract void getRandomIntegerData() throws Exception;

    /**
     * Return the next random integer.
     * @return random signed integer
     */
    public int nextInt() throws InterruptedException {
        logger.debug("Using {} provider", getClass());
        FillQueue fillQueue = new FillQueue();
        fillQueue.run();

        // if taking the last one
        if (integerQueue.size() <= 1) {
            setAvailable(false);
            logger.warn("Pulling the last integer from {} queue", getClass());
        }

        return integerQueue.take();
    }

    /**
     * Thread used to fill the queue asynchronously.
     */
    public class FillQueue extends Thread {
        /**
         * When an object implementing interface <code>Runnable</code> is used to create a thread, starting the thread
         * causes the object's <code>run</code> method to be called in that separately executing thread.
         * <p/>
         * The general contract of the method <code>run</code> is that it may take any action whatsoever.
         * @see Thread#run()
         */
        @Override
        public void run() {
            synchronized (this) {
                if (integerQueue.size() < MIN_QUEUE_LENGTH) {
                    try {
                        getRandomIntegerData();
                    } catch (Exception e) {
                        logger.error("Unable to add integers to queue by thread", e);
                    }
                }
            }
        }
    }
}
Posted in Java | Tagged , , , , , , , , , , | Leave a comment

Filling a queue asynchronously via a thread

I was working with a hardware random number generator (RNG) and what it came down to was we needed a cache of numbers that were available, but also needed the fill the numbers without slowing the process of giving out numbers. I settled on storing the data in a queue. I used a blocking queue to prevent the consumers from ever getting the same number.

private Queue<Integer> values = new LinkedBlockingQueue<>();

When a number was pulled off the queue, I also started a thread that would ensure that the queue was never empty.

    @Override
    public int nextInt(int min, int max) {
        FillQueue fillQueue = new FillQueue();
        fillQueue.run();

        return values.poll();
    }

The thread that filled the queue was an inner-class.

    /**
     * Thread used to fill the queue asynchronously.
     */
    private class FillQueue extends Thread {

        /**
         * When an object implementing interface <code>Runnable</code> is used to create a thread, starting the thread
         * causes the object's <code>run</code> method to be called in that separately executing thread.
         * <p/>
         * The general contract of the method <code>run</code> is that it may take any action whatsoever.
         * @see Thread#run()
         */
        @Override
        public void run() {
            if (values.size() < MIN_QUEUE_LENGTH) {
                try {
                    logger.info("Retrieving {} integers from Comscire", START_QUEUE_LENGTH - MIN_QUEUE_LENGTH);
                    getIntegersFromConscireProcess(START_QUEUE_LENGTH - MIN_QUEUE_LENGTH);
                } catch (IOException e) {
                    logger.error("Unable to add integers to queue by thread", e);
                }
            }
        }
    }
Posted in Java | Tagged , , , , , , | Leave a comment

Apache HttpClient 4 byte handling

After I wrote my own HttpClient ResponseHandler in Apache HttpClient 4 Custom Bytes ResponseHandler, I discovered that there might be a better way. I had used the following utility to get the bytes. I found that one of their classes also used it.

EntityUtils.toByteArray(entity);

Here is what I came up with…

    /**
     * Get the weather radar bytes image.
     * @return byte[] of weather radar data
     */
    @Nullable
    public BufferedHttpEntity getWeatherRadarBytes() {
        HttpClient httpClient = new DefaultHttpClient();
        HttpGet httpGet = new HttpGet("http://services.intellicast.com/200904-01/387694749/Image/Radar/Winter%20Storm%20Mosaic/Loop/SectorName/bwg");


        BufferedHttpEntity bufferedHttpEntity = null;
        try {
            HttpResponse httpResponse = httpClient.execute(httpGet);
            HttpEntity httpEntity = httpResponse.getEntity();
            bufferedHttpEntity = new BufferedHttpEntity(httpEntity);
        } catch (IOException e) {
            log.error("IOException", e);
        } finally {
            // When HttpClient instance is no longer needed,
            // shut down the connection manager to ensure
            // immediate deallocation of all system resources
            httpClient.getConnectionManager().shutdown();
        }
        return bufferedHttpEntity;
    }

An example of using it would be something like the following. NOTE: I never ran this code. Use at your own risk.

    @Test
    public void testGetWeatherRadar() {
        BufferedHttpEntity bufferedHttpEntity = httpService.getWeatherRadarBytes();
        assertNotNull(bufferedHttpEntity);
        assertTrue(bufferedHttpEntity.getContentLength() > 0);

        FileOutputStream fileOutputStream = null;
        try {
            File file = new File("/staticfiles/weather-portlet/kyRadarImage.gif");
            fileOutputStream = new FileOutputStream(file);
            byte[] bytes = new byte[bufferedHttpEntity.getContent().available()];
            bufferedHttpEntity.getContent().read(bytes);
            fileOutputStream.write(bytes);
            fileOutputStream.flush();
            fileOutputStream.close();
        } catch (FileNotFoundException e) {
            fail(e.getMessage());
        } catch (IOException e) {
            fail(e.getMessage());
        } finally {
            if (fileOutputStream != null) {
                try {
                    fileOutputStream.close();
                } catch (IOException e) {
                    fail(e.getMessage());
                }
            }
        }
    }
Posted in Java | Tagged , , , , | Leave a comment

Apache HttpClient 4 Custom Bytes ResponseHandler

An interesting thing I needed was to retrieve an image from a URL. HttpClient provides a BasicResponseHandler, which returns a String. I needed bytes in order to accomplish this, I had to implement my own response handler.

    /**
     * Get the weather radar bytes image.
     * @return byte[] of weather radar data
     */
    @Nullable
    public byte[] getWeatherRadarBytes() {
        HttpClient httpClient = new DefaultHttpClient();
        HttpGet httpGet = new HttpGet("http://services.intellicast.com/200904-01/387694749/Image/Radar/Winter%20Storm%20Mosaic/Loop/SectorName/bwg");

        ResponseHandler<byte[]> responseHandler = new ResponseHandler<byte[]>() {
            @Nullable
            public byte[] handleResponse(
                HttpResponse response) throws IOException {
                HttpEntity entity = response.getEntity();
                if (entity != null) {
                    return EntityUtils.toByteArray(entity);
                } else {
                    return null;
                }
            }
        };

        byte[] responseBody = null;
        try {
            responseBody = httpClient.execute(httpGet, responseHandler);
        } catch (IOException e) {
            log.error("IOException", e);
            //e.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
        } finally {
            // When HttpClient instance is no longer needed,
            // shut down the connection manager to ensure
            // immediate deallocation of all system resources
            httpClient.getConnectionManager().shutdown();
        }
        return responseBody;
    }
Posted in Java | Tagged , , , , , | 1 Comment

Apache HttpClient 4 JSON post example with a generic method

In a previous post HttpClient 4 Json Post I showed how to post a Json object to a URL. That was useful, but required writing a method for each type of object you needed posted. A cleaner way is to use Java Generics to make a method that can accept multiple types of objects and can even return specific objects instead of a Json string.

    /**
     * Returns the http response as the specified object.
     * @param url the url to request
     * @param object the object to convert to the json payload
     * @param responseType the object type to convert the json response into
     * @throws IOException
     */
    public static <T> T postForObject(String url, Object object, Class<T> responseType) throws IOException {
        log.debug("Post: " + url);
        String response = "";
        HttpClient httpclient = new DefaultHttpClient();
        try {
            HttpPost httpPost = new HttpPost(url);
            StringEntity stringEntity = new StringEntity(JsonUtils.toJson(object));
            httpPost.setEntity(stringEntity);
            httpPost.setHeader("Content-type", "application/json");
            ResponseHandler<String> responseHandler = new BasicResponseHandler();
            response = httpclient.execute(httpPost, responseHandler);
        } catch (IOException e) {
            log.error("IOException from " + url + " with " + object , e);
            throw e;
        } finally {
            httpclient.getConnectionManager().shutdown();
        }
        return JsonUtils.toObject(response, responseType);
    }

The method return type is modified to return <T> T and the method signature is changed to include Class<T> responseType. Then responseType is used to tell the Json library what type of object is desired.

Here is a simple usage of the post method.

            AccountResponse accountResponse =
                    HttpUtils.postForObject(Server.IGP_ADAPTER_URL + "/accountBalances",
                            accountRequest,
                            AccountResponse.class);
Posted in Java | Tagged , , , , , , , | Leave a comment

Apache HttpClient 4 JSON post example

Posting JSON to a URL is a common use case. Here is a follow-on to HttpClient Post for a String.

    /**
     * Performs HTTP post.
     * @param url http resource to post
     * @return string response
     */
    protected String post(String url, MyJsonObject myJsonObject) {
        HttpClient httpClient = new DefaultHttpClient();
        HttpPost httpPost = new HttpPost(url);
        ResponseHandler<String> responseHandler = new BasicResponseHandler();
        String responseBody = null;
        try {
            HttpPost httpPost = new HttpPost(url);
            StringEntity stringEntity = new StringEntity(JsonUtils.toJson(myJsonObject));
            httpPost.setEntity(stringEntity);
            httpPost.setHeader("Content-type", "application/json");
            ResponseHandler<String> responseHandler = new BasicResponseHandler();
            response = httpclient.execute(httpPost, responseHandler);
        } catch (IOException e) {
            log.error("IOException from " + url + " with " + object , e);
            throw e;
        } finally {
            httpclient.getConnectionManager().shutdown();
        }
        return responseBody;
    }

Line 13 and 14 convert the object to be posted into a Json string and set it as the payload to be sent to the URL.
Line 15 sets the content type so the URL knows that you can accept JSON as a response.

Posted in Java | Tagged , , , , | 1 Comment

Apache HttpClient 4 String post example

Continuing from my Http Get Example, here is an example of an HttpClient4 String post example.

    /**
     * Performs HTTP post.
     * @param url http resource to post
     * @return string response
     */
    protected String post(String url) {
        HttpClient httpClient = new DefaultHttpClient();
        HttpPost httpPost = new HttpPost(url);
        ResponseHandler<String> responseHandler = new BasicResponseHandler();
        String responseBody = null;
        try {
            responseBody = httpClient.execute(httpPost, responseHandler);
        } catch (IOException e) {
            log.error("IOException", e);
        } finally {
            // When HttpClient instance is no longer needed,
            // shut down the connection manager to ensure
            // immediate deallocation of all system resources
            httpClient.getConnectionManager().shutdown();
        }
        return responseBody;
    }
Posted in Java | Tagged , , , , , , , , | 1 Comment