Using Range Paging in ADF – an example

Introduction:

We recently had a requirement for loading large chunks on data from Oracle table into a background process using Java and ADF framework.  We were not allowed to use “straight” SQL and had to stick to ADF framework!  It took a little bit of time to understand how range paging worked – you see, it was simple to learn how to set the Access mode on a View Object but then it took a little longer to figure out how to use this feature in code. This document is an unpolished look at the lessons learned.

Jumping in:

Querying a lot of rows in a table using ADF View Objects requires a little understanding of how the “Access Mode” works.  There are some great articles online to get you some in depth knowledge on this topic [ See Avrom’s Java EE/Oracle ADF post on this topic and Steve Munech’s blog titled “What is the ADF View Object Range Paging Feature?”] … jump in and read those articles to get your head around what it means.

Lets assume you know what the Access modes are and have select “Range Paging” access mode.  Lets now assume that you have ‘n’ rows (say n=200,000) with a range size of 10,000 rows (I had to read 2 million rows into memory). You will be in essence jumping forward/iterating through ‘p’ pages of 10K rows, where p=20 (200K/10K).

You can see how you can play with the range size number to get to a sweet spot between the number of rows fetched vs number of pages to iterate over.

The rest is simple … you need to ask the framework for the number of pages and then for each page get the rows in range, map this to your data object and viola! Right? Well not quite. When I started using the “getEstimatedRangePageCount()” method, I enthusiastically put this into my loop conditional check – bad idea! Duh!

Lesson Learned: Do getEstimatedRangePageCount() only once.  This method goes through ALL the rows in and tries to figure out the number of pages. Depending on the size of your table … this method can take a while.

Once you begin iterating through the pages, you need to “scroll” to the right page.  I don’t know about you, but my loops always start with ‘0’ and goto “size-1” integer value. So instinctively I said “scrollToPage(p)” where “p=0 to pages-1”. Well not a good idea. … you see page numbers 0 and 1 were the same (check by printing the row values of first and last elements). I don’t know why but scroll to page (p+1) where p went from 0 to pages-1 worked.

Lesson Learned: Always “scroll to page (p+1)” where p goes from “0 to pages-1”

Now there was one other thing that I used to use when using other access mode with Read Only View Objects, I had learned to use the rovo.executeQuery() after binding my variables etc. Naturally, I thought that in the Range Paging mode – I had to do the same, i.e. iterate through the page, do rovo.executeQuery(), ask ADF to explicitly run the query and fetch the data. Actually it turns out,  I was wrong again! We simply needed to “scroll” to the right page and say getRowsInRange for an array of size matching the RangeSize we have specified.

Lesson Learned: No need to do rovo.executeQuery() – scroll to Page and getRowsInRange is enough to fetch the data

Putting it all together

Now lets put it all together. First build your Read only View Object (put in your query, bind what you need to etc) and then goto the “General” tab on the ROVO query and select the right Access Mode and set the Range Size. Here is an example:

Once you have the framework settings done you can code up the fetch in your application module as show below. I use an example that does not need to bind anything (fetches ALL rows) but you can simple bind before you do a getEstimatedPageRowCount.

Code Snippet To Load All Rows in a table into memory:

 9732    public List<MyRowObjectDTO> getAllRowsFromTable()
 9733    {
 9734      QueryAllReadOnlyImpl rovo = getQueryAllRowsReadOnlyRO1();
 9735      List<MyRowObjectDTO> myRowObjDTOList = new ArrayList<MyRowObjectDTO>();
 9736      long st = System.currentTimeMillis();
 9737      int totalPages = rovo.getEstimatedRangePageCount(); //how many pages?
 9738  
 9739      for (int p = 0; p < totalPages; p++)
 9740      {
 9741        rovo.scrollToRangePage(p + 1);
 9742        Row[] rows = rovo.getAllRowsInRange();
 9743  
 9744        for (int i = 0; i < rows.length; i++)
 9745        {
 9746          QueryAllReadOnlyRowImpl curRow = (QueryAllReadOnlyRowImpl) rows[i];
 9747          myRowObjDTOList.add(converEntityObjectToMyDTO(curRow));
 9748        }
 9749      }
 9750  
 9751  
 9752      return myRowObjDTOList;
 9753    }

JDeveloper memory settings

Did you just get the “JDev is low on memory” error? or you have a beast of an application where you are trying to initialize a tree with a million or so nodes? (I was!)

So .. first thought is, its gotta be some “config” file (because I did not start JDev as a jar param to java) … so where could this be?

JDeveloper  jars are under <install_path>/JDeveloper<Major-ver><Release><Minor-version>/jdeveloper folder. In my desktop it was under c:\JDeveloper11R1V2\jdeveloper.

Okay now where? There are two places where we might find the configuration files we are looking for [image 1 – below]. A folder called “jdev” or a folder called “ide” – why the difference Oracle? In any case … the jdev folder has a file called jdev.conf and the ide folder has a file called ide.conf.

Quick look at jdev.conv showed it reference ide.conv and did not have anyline that specified the VM args Xmx and Xms. Finally, looked in ide.conf and found it [image 2 – below].

Imag1 – Folder structure

11gR1 JDeveloper path

Image 2 – ide.conf (notice the path on top)

Design Patterns

Okay so I am officially going to start working through each and every design pattern in the “Sun Certified Enterprise Architect for J2EE Technology” study guide. I keep going through the book from time to time and while it is fun reading UML, there is absolutely no replacement for code and your own experience.

List of design patterns I am going to work on
Design Patterns - Creational, Structural and Behavioral

I started with a top level “design.patterns” package which will have Creational, Structural and Behavioral patterns as a sub-package.  I thought about then writing the examples all under the pattern-type package (design.pattern.Creational) but realized that we may soon have a lot of files from different patterns and not know at-a-glance how many classes we may need per pattern, for example. So I decided to create another level which the Pattern Name so that we can build the Tester and pattern implementation classes under a proper subtree,  “design.pattern.Creational.AbstractFactory” for example.

I also hope to explore some of the anti-pattern implementations we have made and try to reason if that was a good choice then or good choice in hindsight : )

So until then … lets brush up our types (look at the image above – “Creational”, “Structural” and “Behavioral”)  and understand what they mean – i.e. ask your self what is the difference between Creational pattern and Structural pattern and when you would think-about one vs the other.   Good luck!

Matrix manipulation

When solving Boggle, Sudoku or some n-by-n board game or when doing Image analysis or Parallel Computing – it is often useful to be able to represent one-dimensional data structures in 2-dimension and vice-versa.

Simple formulae exists for doing this conversion, as illustrated below:

Converting from Row by Col 2-dimensional (2-index) format to a 1-by-(Row*Col) matrix with only one index:

What we start out with
...what we end up with
...actually its more like this!

Welcome …

Hey!!! Techiecook is ALIVE! Wohoooooo!!!

So what’s in this blog?

  • Recipes
  • Code
  • Tech talks
  • And very random walks through my thought patterns

we will post in this blog – stuff about Food and Technology and whatever comes to our mind!

Alright! Say, did you know that “Kokum” is not the same as “Mace” ? Well I did not …

Kokum - spice
Kokum - The Spice

Mace - Spice
Mace - Spice

I used mace a few times when cooking dishes like “Chicken Shahi Korma” (with and without nuts!) … and the flavour in there was just amazing. I promise to put up the recipe soon!

The first recipe

Hey guess what Sundays are famous for? Omlettes!  As one advert put it – “Eggs love a good omlette” – true. It is also the first dish we are going to talk about because it involves a few basic in cooking.

  • Cooking duration
  • Ingredients and their impact to the flavour
  • Heat (how high your burner is at)
  • And preparation (Chopping/cutting or the lack of it)

So an Omlette can be made in a variety of ways by varying how much you cook your eggs, on how many sides and with how many other things.

Simple Receipe:

Two eggs, some milk (optional), some butter (optional) and some salt and pepper are all that is needed … however if you really want to go a notch up you need to think bigger!

Level 1 – The Garlic Oil:

The very first twist to the simple Omlette is Garlic – yes you Draculas! G-A-R-L-I-C, the white pod thingy (and no I AM NOT TALKING ABOUT SOME DARN PASTE IN A BOTTLE!!!! – Sorry had to scream to make sure you got the point about using quality ) …. anyways here is what you need to do with the GarlicReduce the heat and with a spoon mix the eggs a little so that some of the cooked lumps are incorporated into the uncooked mixture … keep doing this until the beaten mix is half way cooked (i.e. 50% lumpy and 50% fluidy) …then sprinkle some pepper on top.

Remove some garlic cloves from the pod . Smash the garlic with a knife and remove the outer skin. Heat some butter in a pan and throw the garlic in. Keep flame at High. Beat some eggs, add salt.

Watch the garlic and smell – when they turn brown and the aroma of garlic + butter fills the air (yeah that is how I cook) … throw the eggs in the pan.Cover or cook in open and in about 5 mins when you think the omlette is ready flip onto a plate and devour with some toast and butter! Simple yet yummy. Oh yeah and don’t forget the ketchup : )

Level 2 – Milk, Ginger and Chili:

Now cut a small piece of ginger (about 1-2 sq inch),  chop it up fine and then do the same with a couple of green chillis. Add this to the eggs with a little bit of milk when beating the eggs … and then cook the same way as we cooked above. Ginger and chillis add that extra bite to the Omlette and the milk just makes it fluffier.

Level 3 – Vegetables:

Okay its time to get a little extravagant with the dish … if you chop up some Onions (finely) and Green peppers (again chop fine) and umm some Mushrooms … or even a little bit of spinach … you can take these and fry them once on the pan before cooking your garlic and eggs as described above.

I usually start with the butter in the pan, wait till the butter makes “Shwoosh” sound and before it turns brown/black I add the Garlic and cook that … then I add the Onions (use Red Onions if possible) and cook until they are soft and slightly brown. Then I add the Green Peppers and Mushrooms under very high heat … working quickly and tossing my veggies until they are slightly cooked. Finally I add the eggs, beaten with milk and some ginger and chillis. Cook the eggs in low heat, stirring the bottom and sides to fold the cooked portion in until most of it is cooked and then I let it settle. Top off with sliced tomatoes (thin or chunky) and cheese on top (Feta is the best).

Finally  I cover up the whole thing, if possible with an aluminum foil and let it cook slowly for 5-10mins until I can smell the aroma.

I will never add these:

So you experiment and the goal is not just to come up with the next best thing but also to keep good notes of what not to put into an omlette … somethings are just not right for it

– Curry Leaves (I tried it once and the leaves broke the nicely sealed Omlette plus it tasted too much like curry  – my Omlette should smell like eggs and veggies)

– Big chunky vegetables from the frozen section of your fridge Ex: Cauliflower florets and Brocolli florets – if you don’t cook them well enough they will make your eggs had to fit into a sandwich plus they retain the freezer smell (yuck!),

– Chicken (Why?!)

– Soggy tomatoes – dry roast or remove seeds before adding tomatoes to my Omlette please –  but then when you cook its yours so do as you please!

…..

– “Kaali Daal and Chicken” – yes you heard me! It is a bit of an inside joke between my friend and me, and a big lesson learned at the same time.

…..

[ to add more here over time]