May 31, 2010

Caching Framework - Sample Cache Implementation



I was asked in an interview to design a cache framework.
The problem statement was:
Develop a caching framework that provides the following:
1) Load data from file, database or any other source
2) Refreshes the data after a configurable interval if it is not a one time load
3) Invalidates the cached objects after a configurable interval.

Few questions that come to mind while designing is what should be cache.
And If I assume it is going to be Map, then what would be key and value.
Second critical thing is how to expose the Cache to clients.
What operations to support.

I decided for using ConcurrentHashMap as my cache.
There are several benefits of using this Map over others like:
Operations are thread safe supporting full concurrency and highly efficient.

Clients would access the Cache through a CacheProvider.
I created an interface named CacheProvider.
This is the starting point of my Cache Implementation.
Below is the code for CacheProvider:

public interface CacheProvider {
        Object getAndSet( String key);
Object setAndGet( String key );
void refresh();
void invalidateObj();
void cancel();
}


I had an initial requirement where I can invalidate objects in Cache after certain interval if there is no access.
I also decided that,
If a particular data is retrieved then its there should be fresh start of invalidation time, i.e. if I am going to configure the invalidation time as 5 min.
and If some object is not accessed for 5 mins, it would be removed from Cache.
And if some object is not accessed for 4 mins, and some client asks for this object, then there should be fresh start, means it would be invalidated after 5 mins from now, if not accessed in next 5 mins.
I would also be configuring an refresh policy, and refreshing all objects in the cache at particular interval.

Regarding data. I used Derby for testing.
I designed Cache to support data from Database, or from file, or to test simple data can be provided as command line arguments.
I know this is too early to talk about this.
But this is going to be initial requirement of where data would be stored and how MyCacheProvider would load data from data provider.

I created three classes for handling data.
CollectionManager - for managing data from command line
FileManager - for managing data from File
DBManager - for managing data from Database.

I am providing the code for all the data managers.
But if one is not interested, then one can simple ignore the below three classes.


CollectonManager:


FileManager:

DBManager:


DBManager is a class which manages data in database.  I have used Derby.  Using derby is very easy, and its useful in applications like this.  You don't need database, its just one jar. Evrything inside the jar, database, and drivers.


My DBManager class is below:


DBmanager has code for initialization, where it reads config values from a file and intializes a DB connection. 
Then, DBManager has code to create the table, populate the table with data. 
Remember, this data would be used to cache. 
Then code to retrieve all data from the database, this method would be called to refresh the whole cache with data from database. 
And method to retrieve a particular data from database. 




I started my design with a CacheProvider interface. 
Now I would extend that clas and create MyCacheProvider, also I would create a CustomKey to store in Cache.
MyCacheProvider would internally use a ConcurrentHashMap for storing data.


Below is the code of CustomCacheKey, instance of this class would act as key in the Cache, or ConcurrentHashMap inside the MyCacheProvider.

CustomCacheKey:


MyCacheProvider:
MyCacheProvider extends CacheProvider, implements getAndSet() method retrieves the value from Cache, put the same key again in the cahce with updated timestamp.
Whenever a key is retrieved from Cache, its timestamp should be updated, so we either update the timestamp of the key(CustomKey), or put a new key with same value and new timestamp in the Cache.
We have to maintain the timestamp, in order to invalidate inactive objects in Cache.

We also have methods to refresh the cache and append a user provided hashmap to cache.

MyCacheProvider should also contain methods to invalidate a particular object, and provide method to cancel or clear whole cache.


Now, I decided that clients should not directly use the MyCacheProvider, because in future I would be creating some more implementation of CacheProvider, and making it configurable.

I made a CacheHandler, and clients would use this handler, with limited operations exposed.

CacheHandler:


public class CacheHandler {
static MyCacheProvider myCacheProvider;

public static void start(){
myCacheProvider = MyCacheProvider.getCacheProvider();

}
public static Object get(String key){
Object myCachedObject = myCacheProvider.getAndSet(key);
if (myCachedObject == null) {
myCachedObject = myCacheProvider.setAndGet(key);
}
return myCachedObject;
}

public static void refreshCache(){
myCacheProvider.refresh();
}

public static void invalidateObj(){
myCacheProvider.invalidateObj();
}

public static void cancel(){
myCacheProvider.cancel();
myCacheProvider = null;
}

public static Map mapStatus(){
return myCacheProvider.mapStatus();
}
}

For simplicity I made above class a static class.

Now I am done with initial design of Cache with three classes.
MyCacheProvider, CustomCacheKey, and CacheHandler.
CacheProvider is a super interface.
And I created three classes for handling persistent data.


I have to run two tasks periodically:
Invalidate objects after specific interval
Refresh the whole cache.
I decided of one more task, which prints the Cache status.

SO, I am going to create three tasks:
CacheInvalidateObjTask - for invalidating the Cache objects.
CacheMapStatus - for printing the Cache status.
CacheRefreshTask - for refreshing the cache.

CacheInvalidateObjTask:

public class CacheInvalidateObjTask implements Runnable{

public void run(){
CacheHandler.invalidateObj();
}
}


CacheMapStatus:

public class CacheMapStatus implements Runnable{
Map map;

public void run(){
map = CacheHandler.mapStatus();
long time = System.currentTimeMillis();
System.out.println("Map Status.. at.." + time/1000 + " secs..");
System.out.println(map);
}

}


CacheRefreshTask:

public class CacheRefreshTask implements Runnable{

public void run(){
CacheHandler.refreshCache();
}

}


So I created three tasks.

I also created two classes, which can be skipped:

CacheConstants:

public interface CacheConstants {

// Resource provider Constants
String RESOURCE_PROVIDER_COLLECTION = "collection";
String RESOURCE_PROVIDER_DATABASE = "database";
String RESOURCE_PROVIDER_FILE = "file";
}


CacheProperties:

public class CacheProperties {

public static int CACHE_SIZE = 0;

public static long INVALIDATE_OBJ_TIMER_INTERVAL = 1000L;
public static long REFRESH_TIMER_INTERVAL =1000L;
public static long PRINT_MAP_TIMER= 1000L;

public static String RESOURCE_PROVIDER = "";

public static String RESOURCE_FILE_PATH= "";

public static String DATABASE_DRIVER = "";
public static String DATABASE_PROTOCOL = "";
public static String DATABASE_NAME = "";
public static String DATABASE_USERID = "";
public static String DATABASE_PWD = "";
public static String DATABASE_TABLE_NAME = "";
public static Boolean DATABASE_CREATE = true;
public static Boolean DATABASE_CREATE_TABLE = true;
}


To start the application, I have created a class CacheService.
CacheService uses CacheServiceHelper, and delegates the call to CacheServiceHelper.


CacheService:

CacheServiceHelper:


CacheServiceHelper contains methods to schedule the three tasks. And method to set the Env.
And a cleanup method to clear the cache:

void cleanUp(){
CacheHandler.cancel();
}



And our cache is ready.


Summary:

CacheService is the main class to start the program.
CahceServiceHelper is for handling all the startup tasks.
CacheService calls the CacheServiceHelper for initializing the environment, and starting the service.
In initialization, CacheServiceHelper sets the properties, and the resource provider.
Resource provider will provide data to store in the cache, it can be either database, or file, or command line arguments.
CacheServiceHelper, starts the service by creating three tasks and submitting them to a ScheduledthreadPool.
Tasks are:
1. Task to refresh the cache map, periodically, CacheRefreshTask
2. Task to invalidate objects inside cache map after specified time, CacheInvalidateObjTask
3. Task to print the map status to console, CacheMapStatus
Each of these tasks call CacheHandler, which is a fa├žade for all Cache related operations.
CacheHandler, depends on MyCacheProvider for all its operations.
MyCacheProvider is the core class for handling the cache.
MyCacheProvider has a map to hold data. It holds data in a ConcurrentHashMap.

Application needs a cache.properties file with following configurable properties:
intial.cache.size - Initial cache Size.
invalidate.object.interval - Interval to invalidate the object, or remove o0bject from cache.
refresh.cache.interval - Interval for Refreshing the cache.
print.map.timer - Interval to print the contents of cache map.
source - source of the data provider.
Valied values are :
collection – if data is provided via
command Line arguments
database - if data provider is database
file - if data provider is file.


#Database properties
#if database as resource, below properties are mandatory
db.driver=
db.userId=
db.pwd=
db.protocol=
db.dbname=
#optional Database properties
db.createdatabase=
db.createTable=

# File properties.. if “file” is data provider, i.e. source property is file
file.path=C:\\test.txt

Below is the package Structure and UML class diagrams:
Classes:












!!!Any Comments would be really appreciated!!!
Share:

May 30, 2010

Interview Questions when 5 or 6 years of Java/J2EE experience


I recommend reading eBook "Java Notes: Effective Programming & Interview" available on Amazon, and can be read using Kindle or Kindle app.


Generally there were some basic questions:
1. java Objective SCJP level
2. J2EE objective
3. JMS Objective
4. IBM MQ Objective
5. Complicated Multithreading
6. Serialization, File I/O
Below are some extract of questions which I faced while giving Interview of Sapient, Nomura, Gemstone, Mobility, ION, HP, RBS, TIBCO, BOA, CSC. 

Generally I have seen if the interview is for an Investment Bank or any product based organization, then core concepts are very important. And since 2008and just adding this line in 2013 I would say most important concept stills remains Multi threading. And so Executer framework remains hot.

CSC, Chennai
1. 50 numbers ( 1 to 50 ) are distributed in random order in an array of size 50. One number is missing. How to find.
2. Inner Classes, Anonymous Classes.
3. Find manager name who has max. number of employees working under him, when table has employeedID, Name, and ManagerID as columns. and Manager can also be employee.
4. Serialization.
5. How to design a web application, medium size.
6. Write logic to print alternate elements in a ArrayList, even some are removed.
7. How to implement a Hospital scenario, where doctor is examining patients, and emergency cases, and informing doctor of emergency cases, scheduling, etc.
 8. How to select unique elements from a table.
9. Calculate number of legs, and give formula:
7 persons, each person having 7 bags, each bag having 7 dogs, and each dog having 7 puppies.
Bank Of America, Mumbai
1. How is semaphore implemented.
2. How to ensure only one instance in Singleton class.
3. How synchronization works, inside JVM.
4. Master JVM is holding objects. Slave JVM's are reading and writing the object.
What problems can come and what is solution.
5. How Garbage Collection works in JVM.
6. When are class garbage collected, when are classes unloaded.
7. How to ensure that instance is never garbage collected.
8. Different protocols for Web Services.
9. Managing Session in Web Services
10. RMI, how EJB works
11. passing objects between two JVM's.
12. role of WSDL, stubs, etc. in Web Services.
13. Garbage Collection Tuning parameters.
14. LAZY Loading in ORM.
15. Why there is Stack and heap in JVM. Why this structure.
16. how much memory a program consumes, when no objects created inside program.
1. JCA - Tntro
2. JPA - Intro
3. EAGER/ LAZY loading
4. JPA relations
5. IBM MQ Objective
6. JMS session, Thread Safety In JMS
7. JMS providers
8. JMS implementation
9. AQ Provider and implementation using JNDI
10. Garbage Collection algorithms.
11. SOAP
12. JTA.
RBS, Gurgaon
1. Hashing Algorithm.
2. Search Algorithm
3. MVC design Pattern.
4. Multi threading Objectives.
5. Design Application which handles a particular operation for millions of trades, operation like trade validation. Application communicates external application for data.
6. Several Designing Questions.
7. Scala, Open Source high performing JMS implementations, Dynamic languages
8. Cache Implementation.
9. How Oracle Coherence works.
Gemstone, Pune
1. Why Use EJB 3.0
2. Why use JPA
3. Why use JPA with ORM, Standard ORM products
4. Writing Complex Query while using JPA and ORM
5. JTA, how it works
6. JTA, how to use
7. Where to use JDBC and where ORM
8. JSF Architecture, who is model, view and controller
9. Optimistic Lock in JPA - Detailed
10. Multithreaded program to read large file/ code and design
HP Labs, Banglore
1. Latch, Barrier, synchronized, Lock, ThreadPoolExecuter, Garbage Collection Algo.
2. Which forums do you follow,  Cloud Computing, How to improve programming and Coding skills.
3.  Design horse racing application.
4. Struts, how to extend struts. Why to use Struts.
5.  Where to use messaging
1. Core Java SCJP level
2. Current Project Architecture
3. Message Flow in current project.
4. Fault Tolerance in JMS
5. Hashing Algo in Java, HashMap
6. Aggregation, Composition, Encapsulation, Abstraction.
7. How to implement Thread Pool
8. Complete Garbage Colection Algo.
9. DataSource, DB connection pool - how it works.
10. Exception- Logic, how to handle and design Exception, Error.
11. NIO-concept, Socket handling using NIO.
12. Adapter Design pattern
13. Stored Proc, Trigger
14. JDK1.5 Thread pool
15. Serialization, Externalization
16. Immutable, FInal, Usage of Final and Immutable
17. Exception handling in the spawned threads
18. RESTful Web Services, JEEVAN
19. Types of Parsers
1. deep copy and shallow copy
2. hash code
3. graceful shutdown of threads
4. NaN, 0/0
5. wrapper classes.
6. BootStrap Class Loader
7. Oracle OCI.
8. Oracle JDBC drivers
9. Oracle Thin and Thick Clients.
10. Oracle Rak
11. MQ API, MQ classes for Java.
12. MessageId, Priority for MQ message.
13. Garbage Collection
14. Java Green Threads
15. Coorelated Sub Queries.
16. Steps to make connection to MQ using Java.
17. Options to set for browsing message in MQ Queue.
18. Client Mode/Bindings Mode.
19. Log4j
1. How would you implement your own Thread Pool.
2. What is NIO
3. Best way to read data from socket using traditional I/O
4. Beat way to read file.
5. How to read characters from InputStream.
6. How Serialization works
7. Synchronization
8. how to implement counter in servlet.
9. how hashtable is thread safe.
10. CharSet, Enocder, Decoder
11. AtomicInteger, volatile, transient.
12. JTA, XA, how XA works, two phase commit.
13. Role of transaction manager in XA transaction.
14. XADataSource, and closing XADataSource.
15. UserTransaction
16. JCA Architecture
1. How would you implement cache in java.
2. How would you implement auto complete using java, jsp.
3. How would you implement a chache with auto complete feature.
4. How to make class immutable.
5. How to serialize immutable class.
6. How to gaurantee message delivery in JMS, fault tolerance
7. Message recovery techniques and redelivery.
8. DROP table.
9. Oracle Stored Procedure or function
10. Struts architecture
11. JSP Exception Handling
12. java Objective SCJP Level
13. java Design Pattern
14. J2EE objective
15. Implementing references
16. UNIX Commands - grep, softlink, hardlink
17. Serializable, Externalizable, Cloneable
JDk 1.5 Collections classes.
Multintreading Concepts - Barrier, Latch, etc.
JMS functioning
SQl Query using JOINS
Java Design pattern - detail
Overloading, Overriding, Comparator, Comparable, hashCode(), equals(), Serializable, Externalizable, RMI, garbage Collection.
Please read my more posts in the same category - Answers Java/J2EE interview questions
Share:
© Shift, ShEkUP, Shape, and Surprise | All rights reserved.
Blogger Template Crafted by pipdig