Predictive Cloud Computing (PCC) for professional golf & tennis: Open JPA #2
Articles,  Blog

Predictive Cloud Computing (PCC) for professional golf & tennis: Open JPA #2


Hi welcome back to the second part of the Open Java Persistence Application programming
interface that we used within the Predictive Cloud Computing
for professional golf and tennis events. I’m Aaron Baughman. I work for IBM’s Continuous Availability Services. I want to focus in on our Persistence Project
and in particular look at some of the Data Access
Objects and their ID classes and then talk about just
a couple of the annotations within the classes. Then from there we’ll go into the Persistence
Loader Project where we can talk about some of the different
types of persistence loaders as well as the persisters, and after that we’ll look at the Big Engine
Project where we can look at how we open up resources
and make sure that all of the resources are closed
as required so we don’t have any memory leaks which is
quite common. And then I would like to look at the UIMA
framework which is the Unstructured Information Management
Architecture asynchronous scaleout part where we also use
a lot of these different database connections. So first if we go back down and we scroll
to the Persistence Project, let’s take a look at
the PlayerDAO, ok. So the Player DAO, located here, open it up. This is a standard Java class, right, where
we want to go ahead and import the different libraries
in which we need. So you see the javax.persistence from the
Persistence API have been imported and then we also have imported
the apache.openjpa, some of those objects, and then right after that we then followed up with the class definition so we say this is the class of the PlayerDAO. We do want to implement a Serializable so
we can serialize out objects as required. And then we do implement our own interface
that we’ve defined called Player which I showed you in
part one. And we also map for the ORM, the table name
is Players and the ID Class that we use to look up specific players from DB2 is the PlayerId.class. And as we go through and we define each one
of the instance variables such as the Player ID, we define that yes, this is an ID column. And if you look and you see multiple ID columns, then it’s called a Composite Key. So we have two ID columns which compose a
Composite ID, but then if you look in the SiteDAO, the SiteDAO has its own ID classes which perhaps I’ll show you that. But you can also see that I name the playerId instance variable column name the table ID
and I give the length of 20 here. And then we get into the annotation of Many
To One and this says that many players can belong
to one specific site and this is the @ManyToOne relationship. And generally JPA produces a joint table for
us. And if we look at the set which says I can only have one value of a particular type within
a set so they’re a unique list of player types. This is an Element Collection and it is a
type enumeration because the player type is in a numeration, and let’s go ahead and look at the Player
Type. So I go there and I can see that all this
class is is a enumeration, where I have a Player Type
and two types exist, Preliminary and Tournament. So whenever I look at this class I can see
that if I want to get a Player Type by the actual
name, Preliminary, Tournament, that I pass in after
the static method here, defining method, I pass
in the player type that I want and then I use the
shortcut within Java passing the values, which then iterates through Preliminary and Tournament
with the pt variable and then I get the name which
would be the value for the Preliminary or Tournament, and I want to just make sure that it does in fact equal the type, and if it does then I want to return that kind of Player
Type. Otherwise I’m going to go ahead and throw an exception, an Illegal Argument Exception
because we don’t know that kind of player type has
been passed in since it doesn’t match Preliminary
or Tournament. Likewise I can do the same with getPlayerType
if I want to pass in the site information as well as the current time in which we’re on and
the current time could be a replay time if we’re
going back in time to forward into a simulation or it could be the actual real time, time. And so I pass it in, right, and I call in
site. I want to check to see for this site object, is this a Preliminary time or not. And if it is then of course I’m going to return Player Type of Preliminary, otherwise Tournament
type. Now, just to show you, whenever I connect
to DB2. Alright, so I’ve already connected but I can
show you here’s the commands that I use. I went ahead and I pseudoed over into the
administration ID for DB2, and then after I did that, after I pseudoed into it, which is called
eibluedb then I would go ahead and issue the command
DB2 which then in turn brings up the DB2 console. And here is where I issued the select distinct player_types from eiblueus schema .playerdao_playerTypes and playerdao_playerTypes, that is an automatically
generated table from JPA because of that ManyToOne annotation,
right, and if I go ahead and execute this command
then you can see that I have Preliminary and Tournament within the database itself, right, so it does you know do a nice mapping for
me. Right, so that’s the Player Type, and let’s
look at the PlayerId so you can get a better idea as far as how the ID classes work, because if you recall up here, right, that I have Players as a table name, but then the IdClass is the PlayerId.class. So if I go to a Player ID then I need to have my String playerId which then maps over to my ID column here, playerId, and likewise I need to have a site as well. So I have site and playerIds, they both exist. And I just have you know of course my constructor, I pass my playerId and my site and I set the
two instance variables through that constructor, and then I have an empty constructor to support
JPA and I can go ahead and return. So if I want to create a new player from a supplied player and site then I can go ahead and call from Player, and then I have the
standard equals, right, your hashCode and your toStrings so that if I want to debug then I can go ahead
and print out all of the different Objects, so I do add Player ID, playerId and so I know
that a verbose description of the instance variable
that I’ve printed out is there, and then your hashCode and your equals need to be consistent since they are used interchangeably. Let’s also look at another class. We can look at the Twitter class here, so let’s go through and look at the ReachDAO to give you another example. And here within the Reach again, this is an
entity. The table is going to be Reach, and again I have my ReachId.class so I can go ahead and look up any kind of Reach records or rows within a table given their ReachID, and I then in turn say ok my column is going to be Instant, so it maps over to this instant, and likewise reachType is an enumeration so much like playerType, the reachType is also, and let’s just look at that again, right,
and all it is is it has three different kinds
of reaches. So we have Tournament reach, Player reach, and then All reach. And let’s define what reach is. So reach looks at, you know, player clout, it looks at the number of followers, it looks at the acceleration of tweets by
a user, and it also looks at retweets, and it calculates what is the influence that a user has during a tournament, right, and we calculate
that and then store it within DB2. And this is what this reach is. And if we look at HistoricalLogCount here, what happens is we have a Hadoop job that runs at 1am cut time and it correlates together the web crawler, so the web crawler goes off
onto one of the sporting websites such as the Australian
Open and it finds out on which page which players are mentioned and at specific times, and then we store that within the database, and then we in turn correlate that with log entries so as people access our cloud then within our IBM HTP server we then log
who is accessing what page at what time. And with that information we can figure out how many of the webpages are being accessed about a player at any given time. Alright so that’s one Hadoop job that runs and we also have historical log entries that
also runs which goes back and it looks at all of the different log entries and for every minute it bins how many times or how many hits we’ve had per minute, and we save it within our historical log entries here. So we have our site, which again many, so many of our log entries, historical log
entries, belong to one site and we have a timestamp as well as a count, how many of those, how many hits do we have at that time for a particular venue. Alright, so that’s how we in turn save that. Now if you go down to a different project, the PersistenceLoader, I wanted to show you
Players and this, the purpose of this is we want to extract the function or usage of the DAOs from the actual loading and persistence. So the function of the players is located
here. So within this class I up front define a function, and this is a guava function which in Java
1.a, but we’re able to pass in different functions, references to functions, with inputs, and
then we can go ahead and get whatever output we
would like. So in this one here we say that within this
function we want to have an input of a PlayerDAO, and we want to have an output of Player so
that throughout the code we’re not passed around
a PlayerDAO but we’re passing around the Player interface. Right and so what I do is I take in as input the PlayerDAO and then this is pretty much like a casting mechanism where I just pretty much return the input and it becomes
a player so that I in turn have a list of players that I can produce. And I’ll show you where it is actually being
used, so let’s find it within this class here. So here I do a Lists.transform and this transform takes in a list of results from a query and
it takes in a function, and it combines the function over every single object in this list so that
we can return a list of players instead of a list
of DAOs. So that’s the purpose of doing that. And then of course I set my logging, so I
set my LoggerFactory and I’m able to control what level of logging do I want to output from
this class. And here I have a getPlayerMap. This is a cashing mechanism so that I can
get all players at certain intervals and put it
into a HashMap so it’s much faster for lookup for
later. And then this is the actual getAllPlayers,
right. So we don’t use any straight JDBC code at
all, but we go ahead and use the JPA mechanism
of code. So the first item that I do is I pass in a
manager, an entity manager, which is a JPA manager, and I want to create a builder. So once I have this criteriaBuilder I then create a query and the query is going to be on this string class here. And then I also have a root, playerRoot, and the playerRoot tells me what table or
what object, which then maps to a particular table, do
I want to query from, and this is from the playerDAO.class which is from the Persistence class. And then I issue my select statement, so I
do playerRoot.get and I want to get all of the playerIds from the PlayerDAO.class and hence I’m following the String.class here, and then I finally issue a query here. So my query is going to be a type query of
String, and I create that query with the criteria that I selected and then I actually get the
results out of the query and return it as an ImmutableList so that the content within that list cannot
be changed. And then just to show you another example
here of getAllPlayers except this one has site
conditions where I can add in “and” right and I say that my equals condition the PlayerDAO site.name must equal the site name that I passed in. And I do the same for the site year, the site year must equal the site.getYear
that I pass in. And that is how I start building up these large, complex site conditions, and I can
also set my criteria.select to the playerRoot which comes
from here so my PlayerDAO.class, and then I say where
which is given my siteCondition so I can set the balance
of the rows I want to go ahead and retrieve back from the database. So I create the query and then I go ahead and get the results and then I transform the
results with the function that I showed you up above so that I return a list of players and not a list of PlayerDAO. And then this is another getPlayersForIDs
where if I pass in an iterable of strings which
are player IDs with sites, and then I want to go ahead and transform that into again just a player, a set of players in this case. And it’s of note to note that many times you might try to get a player from a database, in our case a computer project, but it may
not exist and so if it doesn’t exist I want to go ahead and return an optional.absent. I don’t want to return null or anything like
that. So I do optional.absent if I can’t look up
the player with the id, and I look it up from the entityManager. I do entityManager.find and I say I want to
find it from the PlayerDAO.class and then I create
a new PlayerId that has the id and the Site
information that I showed you before. And then I have some other classes here where I want to see do I have a valid PlayerID or
not, and I just issue again these different types
of selects, select my PlayerId given my playerIDCondition where I say my PlayerDAO playerId should equal
an ID and then I say, just to be quicker about the
sequel, I say I really only want one result. Just give me the first result that you can
find, alright, and whenever I have that then it
says get the single result as long as it’s greater
than zero then it’s going to be true which says yes, I in fact did find a match, otherwise I did
not. And then reversely if I want to take a player
object or interface here and then I want to convert
it over to a DAO, then I have Preconditions. So it’s good to always have Preconditions
in front of every single one of your methods so you
can make sure that the various settings are not
null such as I want to check that entityManager
is NotNull and the site is NotNull. You can also check ranges and values as well. Up here I do player instance of PlayerID,
print, so I can go ahead and cast it over. If it’s not a PlayerID then I need to go ahead and see if I can find that player from the
database and if so, then I cast it over. Otherwise if I can’t find it then I say it’s
absent. And that’s how we do some retrieval of different
players. Let’s look at Rounds now. So Rounds is quite similar, where I define
a function up front, so the following functional programming parallels with pass in function of the methods. But I wanted to show you here some more complex type playerConditions where I say
I want to get golf rounds only for the kind of player. So I say I want to query the RoundDAO.class which again translates to a table, and the site condition I want to make sure that the site year and the site name are equal from what I passed in, and then I also want to check to make sure that the playerId is equal to what I had passed in for the player
so I’m getting all relevant pulls just for that
player. And I use “and” to nest together different
types of predicates. And finally I set the criteria .where to player
condition, and I go through and I set up my query given my criteria that has my select, it also has my where clause, and then I also issue a filter where I make sure that I don’t have any empty holes. Any empty hole means that the hole does not
have a score yet, alright so I only want holes
that are complete. And then I go ahead and I transform the filteredRounds into a list of Rounds and then I return it. And let’s also look at FeaturedGroupLoader
very quickly, and this is where I load different featured
groups from the table based upon the time period that we’re on as well as the event. And this is for golf only. So I’m going to look at the getClosestFeaturedGroup which is down here, and you know it follows the same kind of pattern but what I wanted
to show you here was I have an orderBy. So I do criteria.orderBy and I say I want
to, I want to order the criteriaBuilder by descending from the groupRoot.get time, so I want to
have a descending time order from the groupRoot, so from FeaturedGroupDAO.class. I want to make sure that for my criteriaBuilder
here that I have my “and”, that all of that will
be ordered so that I can make sure that I get
the closest featured group to the nearest time. And that’s how you can do that. Now, when I go on and I want to use those
two projects right, let’s look at the Big Engine class. The Big Engine project is what is drives this application through Quartz multi-threading
which we’ll talk about in the second tutorial. But let’s look at the golf scores job. The GolfScoresJob, what happens with this
is it parses in a scores file every couple of
minutes and then every couple of minutes it then parses
it in and then it saves it to the database, we have accurate scores for every round and hole the player needs to play, ok. And this is the class name, and we extend AbstractBigEngineJob which is a Quartz job, and then we override the execute, right, where we get in a TournamentConfiguration
website which follows a context pattern and then we get a JobExecutionContext as well. And this calls calculateScores, and calculateScores in turn gets that TournamentConfiguration
where we can get the site information and the time
zone so you set the appropriate time zone or time for the event in which we’re on, and we can
set the simulation time, the start time, and end
time that we care about of this time. And here is where I create a manager from
the configuration so I do config.createEntityManager, and let’s look at this createEntityManager. So it’s abstract which means that there’s
a class that actually has to implement it. Let’s close that and let’s double click on
this. Let’s look at the implementation. And this is the implementation, right, where I do factory.createEntityManager. And let’s follow the implementation of that. Right and then this is where we actually get
into, start getting into the actual API where the libraries create these entity managers for
us. So let’s close that and close this class and continue on and see where we use the actual
manager. Ok, so I use the manager here, so I want to get the Rounds for a period. So I say get me all the rounds from the time that I’ve set here in milliseconds to the
next day. And the time is going to be start time, ok. And we iterate through, and we get all of
the start times for the round as long as we’re before the end. And I go through and I say, ok I want to go ahead and get the player map or create a new player map, right, and then I get the player map here, .get, and I say get me the round for that. And I continue through the logic, and I say get players, .getPlayerForID and pass in the manager that I opened before, I pass in the site and the playerID, and then I want to make sure I can go ahead and get the ScoreResponse from the player, and then I can put that into the playerMap,
ok. And then finally I have a finally block here we’re going to try. And I make sure that I close every single manager resource, otherwise you’re going to have many leaks. And the playerMap is very important because
the playerMap allows me to save a lot of time as I go through each one of the round objects. And that’s how it works from the Quartz perspective. And then if you go and look at UIMA, alright so let’s look at the factors. The Factors is a way of which we distribute
algorithms across many different JPAs, and some of the algorithms, or one of them that I wanted to
show you was the TennisLogRealTimeFactorPipe, which is this one here. And let’s look at line 72. See here is where I create a factory. So here I have to create a factory first before I can create the EntityManager. So I say create an EntityManager given the PersistenceUnit which maps to my Persistence.xml
file. And let’s look at how we do that. So I pass in that persistenceUnit, and I say ok, create this EntityManagerFactory. And again we’re starting to get into the specific implementation for the JPA provided. Let’s close that and go back. So once I have the factory, I say factory go ahead and create me an EntityManager, and now I have my manager that I can use to look
up. And for example I’m loading a site with a
manager but I also pass in a name and a year so I have that site. And then if I go all the way down towards the bottom then I have to make sure that I close the manager but also close the factory so I don’t have any memory leaks. And UIMA really shows you if you have memory
leaks because it’s called thousands and thousands
of times during the duration of the project. So every time this is called it would still hold on to a reference to a manager at a factory, and it would get worse and worse. So that concludes part two of the JPA tutorial.

Leave a Reply

Your email address will not be published. Required fields are marked *