Showing posts with label getitem. Show all posts
Showing posts with label getitem. Show all posts

Saturday, 2 May 2015

Performance of getitem

I've previously posted about how scanning is slow, and I stand by that.  But I've recently discovered that in some situations, it can be less slow than using getitem.  

I was doing a code review of some old code and found that it was using $scan in a situation where I thought you'd usually use getitem, and wondered why someone would have done it this way.  But before I replaced it, I wanted to check the performance to see what difference I would be making by changing it, and I was surprised by the results!

The code was designed to check if a variable matched one of a reasonably large number of items.  For this example, if stuck with 10 items...

  if ( temp = "ONE" | temp = "TWO" | temp = "THREE" | temp = "FOUR" | temp = "FIVE" | temp = "SIX" | temp = "SEVEN" | temp = "EIGHT" | temp = "NINE" | temp = "TEN" )
    ;testing condition
  endif

I could have used a line continuation marker, but you get the idea.

What the developer had done is replaced this set of conditions with a single $scan, like this...

  list = "|ONE|TWO|THREE|FOUR|FIVE|SIX|SEVEN|EIGHT|NINE|TEN|"
 
if ( $scan(list,"|%%temp%%%|") > 0 )
    ;testing condition
  endif

Note that this is not a Uniface list, a "bar" or "pipe" character has been used as the delimiter - this is placed at the beginning and end of the value to ensure it is not found as a sub-part of another longer value.  It This takes up a lot less space, and is perfectly readable, but I was concerned about performance.  

To test this, I wanted to make sure it was fair, so I decided to always check both the first and the last item in the list in each iteration.  I also wanted to test in a few different ways, and I came up with 4...

1) Set of conditions

  temp = "ONE"
  if ( temp = "ONE" | temp = "TWO" | temp = "THREE" | temp = "FOUR" | temp = "FIVE" | temp = "SIX" | temp = "SEVEN" | temp = "EIGHT" | temp = "NINE" | temp = "TEN" )
    ;testing condition
  endif
  temp = "TEN"
  if ( temp = "ONE" | temp = "TWO" | temp = "THREE" | temp = "FOUR" | temp = "FIVE" | temp = "SIX" | temp = "SEVEN" | temp = "EIGHT" | temp = "NINE" | temp = "TEN" )
    ;testing condition
  endif

2) $scan a bar delimited string

  list = "|ONE|TWO|THREE|FOUR|FIVE|SIX|SEVEN|EIGHT|NINE|TEN|"
 
if ( $scan(list,"|ONE|") > 0 )
    ;testing condition
  endif
  list = "|ONE|TWO|THREE|FOUR|FIVE|SIX|SEVEN|EIGHT|NINE|TEN|"
  if ( $scan(list,"|TEN|") > 0 )
    ;testing condition
  endif

3) getitem/id a Uniface list

  list = "ONE·;TWO·;THREE·;FOUR·;FIVE·;SIX·;SEVEN·;EIGHT·;NINE·;TEN"
  getitem/id temp,list,"ONE"
  if ( $status > 0 )
    ;testing condition
  endif
  list = "ONE·;TWO·;THREE·;FOUR·;FIVE·;SIX·;SEVEN·;EIGHT·;NINE·;TEN"
  getitem/id temp,list,"TEN"
  if ( $status > 0 )
    ;testing condition
  endif

4) $item a Uniface list

  list = "ONE·;TWO·;THREE·;FOUR·;FIVE·;SIX·;SEVEN·;EIGHT·;NINE·;TEN"
  if ( $item("ONE",list) != "" )
    ;testing condition
  endif
  list = "ONE·;TWO·;THREE·;FOUR·;FIVE·;SIX·;SEVEN·;EIGHT·;NINE·;TEN"
  if ( $item("TEN",list) != "" )
    ;testing condition
  endif


I wasn't really sure what I was expecting, but I thought the $scan would be the worst performance.  I tested over 2,000,000 iterations, and here's what I got...

1) Set of conditions: 37.30, 36.95, 37.83 = 37.36 secs
2) $scan a bar delimited string: 21.76, 21.97, 21.24 = 21.66 secs
3) getitem/id a Uniface list: 24.46, 24.22, 24.89 = 24.52 secs
4) $item a Uniface list: 22.65, 23.25, 23.47 = 23.12 secs

So the slowest was the set of conditions.  I didn't test it, but knowing that if statements shortcut I figure that if the item was always passing the first condition it would be quick, but because half of my test items were only passing the last condition, it would have to check each of the conditions in the set before it passed.

Although there wasn't a big difference, what surprised me is that the getitem/id and $item were actually slower than the $scan in this case.  I then remembered back to a conversation on the Uniface-L mailing list, which talked about how Uniface handles lists in the background.  The description there indicates that an array is built in the background, which means there is an upfront cost for calling getitem/id (or $item) once, but then if you're looping through it is much quicker to access the rest, because the array can be used.  However, because I'm rebuilding the list each time, that means the array needs to be rebuilt each time.  

This means that actually $scan can be used to improve the performance when checking that an item is in a list, as long as you're only checking this list once and it's not going to be re-used.  I expect there to be a point at which the number of times the list is re-used means that using getitem/id (or $item) would become better for performance.

Also, I've not tested different lengths of lists.  However, given the results and my reasoning for why the results ended up this way, I would have thought extending the list would simply emphasize the results.

Summary:  Checking if an item is in a list can be done a number of ways, and if it's being done a lot of times, performance can be eeked out by using a $scan, surprisingly!

Thursday, 19 July 2012

Types of for loops - part one

I have already discussed the basic for loop in my last post, but in Uniface 9.5 there were a number of other list constructs made available, which I plan to investigate over the next few posts, having never used them before.  


I wrote a post a couple of months ago entitled Performance of list processing, which looked at different ways of looping through a Uniface list of values.  In this post I determined that one of the quickest ways was a while loop with a counter, using getitem to extract each value in turn, something like this...


  count = 0
  $status = 1
  while ( $status > 0 )
    count = count+1
    getitem temp,list,count
    ;do something
  endwhile


However, one of the new constructs is forlist, which can be used to the same affect...

  forlist temp,count in list
    ;do something
  endfor

As you can see, the code is much more concise.  There is no need to initialise the count variable or  $status, everything is done as part of the forlist statement, and the incrementing and extracting are done automatically.  The "count" variable is optional, if you don't need it then you don't need to include it.


It is also possible to do the same thing when you have an ID list, where the same construct will return both the ID and the value of each item in the list separately...


  forlist/id id,temp,count in list
    ;do something
  endfor

The "count" variable is also optional in this case.  You know what's coming next...

So let's test these three blocks of code over 2,000,000 iterations...

  • while = 01:18.47, 01:17.55, 01:18.11 (around 1 minute 18 seconds)
  • forlist = 01:08.06, 01:08.51, 01:08.42 (just over 1 minute 8 seconds)
  • forlist/id = 01:17.27, 01:18.03, 01:17.26 (just over 1 minute 17 seconds)

As you can see, forlist is not only more concise from a coding perspective, but it also performs better.  Given the number of iterations, the performance gain would probably be limited, but it is clearly the better option.

Summary: Whilst I have previously always used while loops, I shall now be considering switching the forlist loops, for iterating through a list.  

Friday, 18 May 2012

Performance of list processing

As mentioned in my previous posts, performance is a key factor.  The lovely Uniface list is effectively string a delimited string, as you know, which means we're back to string manipulation, which we know is costly.  Unfortunately there are no arrays or similar available, so we make do.


As far as I can see, there are four ways to process a list....

  1. getitem with a counter

    count = 0
    $status = 1
    while ( $status > 0 )
      count = count+1
      getitem temp,list,count
    endwhile

  2. getitem/id with a counter

    count = 0
    $status = 1
    while ( $status > 0 )
      count = count+1
      getitem/id temp,list,count
    endwhile

  3. getitem with a destructive list

    count = 0
    while ( list != "" )
      count = count+1
      getitem temp,list,1 
      delitem list,1
    endwhile

  4. getitem with a reverse destructive list 

    count = 0
    while ( list != "" )
      count = count+1
      getitem temp,list,-1
      delitem list,-1
    endwhile
These loops all use local variables and the list is pre-built with 20,000 items like...

"1= ABCDEFGHIJKLMNOPQRSTUVWXYZ 01234567890 abcdefghijklmnopqrstuvwxyz 01234567890"

  1. 00:06.58, 00:06.54, 00:06.69 (just under 7 seconds)
  2. 00:06.85, 00:06.89, 00:06.84 (just under 7 seconds)
  3. 01:02.16, 01:01.87, 01:01.91 (just over 1 minute)
  4. 00:06.61, 00:06.63, 00:06.72 (just under 7 seconds)
There is obviously a clear loser here, never do a destructive list!  I think the main reason for this is that the getitem must have to scan through the string, but the delitem is then causing the whole list to be rebuilt - very costly!  

Now I shall try again but with a pre-built list of 200,000 items, the same as those before...

  1. 01:06.55, 01:06.38, 01:05.99 (just over 1 minute)
  2. 01:19.22, 01:19.88, 01:19.36 (around 1.3 minutes)
  3. -
  4. 01:05.38, 01:06.59, 01:05.45 (just over 1 minute)
This shows that the next worst is the getitem/id, whereas the remaining two are pretty close.  The reason I think the reverse destructive list works fairly well is that the list is being shortened each time, which means there is a smaller string to scan.  However, finding the first item must be quicker than finding the last, implying that the scan always happens from the first character.  What this probably means, although the numbers are too close to really prove this, is that the reverse destructive starts to perform better for really really long lists, but the plain getitem performs just as well for shorter lists.

Summary: When processing a list it is best to use either a getitem with a counter or a reverse destructive list, as these both perform well for normal sized lists.