One common use for the while loop is to loop through a set of retrieved records. This can also be done with a for loop, as I discussed in a previous post about types of for loops. The example I gave was something like this...
This is nice and simple, and also pretty efficient. There is one big downfall with it though, and that's memory. In order to improve performance when you do a retrieve, Uniface works out what the "hitlist" is, but doesn't actually pull all of the retrieved records into memory. As you loop through, more and more of the hitlist is completed, and the records are pulled into memory.
To demonstrate this, I retrieved 600 records and then looped through them, using Process Manager to track the memory of the "uniface.exe" program...
As you can see, retrieving the hitlist is just the first step, and a small one at that, the data is not read in until you loop through the occurrences.
If you had retrieved a rather large amount of data, looping through it like this would continue to hold all of that data in memory. In the situation where you're processing records one by one, you often no longer need the record after it's been processed, so it's a good idea to discard the record, to free up the memory.
Each time you discard a record, it removes it from the hitlist and then does an implicit setocc to the next record, which becomes the new current record. Another thing that discard does is set $status to be the occurrence number, or 0 if there are no records left in the hitlist, just like setocc does. This allows us to build a very similar loop to the one we had before...
An alternative that I often see is something more like this...
endwhile
I'm not a fan of this one though, as $hits can sometimes have performance problems of it's own, caused by it's tendency to complete the hitlist in order to return the count.
So, let's check how these different methods stack up against each other performance-wise, using the same 600 records as before...
Summary: Using discard with a large record set is a good idea, because it reduces the memory footprint of the userver, and at no noticeable cost to processing time.
setocc "ent",1
while ( $status > 0 )
;do something
setocc "ent",$curocc(ent)+1
endwhile This is nice and simple, and also pretty efficient. There is one big downfall with it though, and that's memory. In order to improve performance when you do a retrieve, Uniface works out what the "hitlist" is, but doesn't actually pull all of the retrieved records into memory. As you loop through, more and more of the hitlist is completed, and the records are pulled into memory.
To demonstrate this, I retrieved 600 records and then looped through them, using Process Manager to track the memory of the "uniface.exe" program...
- Initial value: 14.56Mb
- 600 records retrieved: 14.64Mb (+0.08Mb)
- Looped to occurrence 100: 14.91Mb (+0.27Mb)
- Looped to occurrence 200: 15.30Mb (+0.39Mb)
- Looped to occurrence 300: 15.70Mb (+0.40Mb)
- Looped to occurrence 400: 16.10Mb (+0.40Mb)
- Looped to occurrence 500: 16.54Mb (+0.44Mb)
- Looped to occurrence 600: 16.93Mb (+0.39Mb)
As you can see, retrieving the hitlist is just the first step, and a small one at that, the data is not read in until you loop through the occurrences.
If you had retrieved a rather large amount of data, looping through it like this would continue to hold all of that data in memory. In the situation where you're processing records one by one, you often no longer need the record after it's been processed, so it's a good idea to discard the record, to free up the memory.
Each time you discard a record, it removes it from the hitlist and then does an implicit setocc to the next record, which becomes the new current record. Another thing that discard does is set $status to be the occurrence number, or 0 if there are no records left in the hitlist, just like setocc does. This allows us to build a very similar loop to the one we had before...
setocc "ent",1
while ( $status > 0 )
;do something
discard "ent",$curocc(ent)
endwhile
An alternative that I often see is something more like this...
setocc "ent",1
while ( $hits("ent") > 0 )
;do something
discard "ent",1
endwhile
I'm not a fan of this one though, as $hits can sometimes have performance problems of it's own, caused by it's tendency to complete the hitlist in order to return the count.
So, let's check how these different methods stack up against each other performance-wise, using the same 600 records as before...
- Original loop: 6.37, 6.02, 5.90 (around 6 seconds)
- Discard loop with $curocc: 5.95, 5.95, 5.86 (just under 6 seconds)
- Discard loop with $hits: 5.95, 6.00, 5.89 (just under 6 seconds)
As you can see, there isn't that much difference in the approaches as far as performance goes. In this case $hits hasn't caused a problem, but if I remember correctly, it's environment and/or database specific, so that's probably why.
Summary: Using discard with a large record set is a good idea, because it reduces the memory footprint of the userver, and at no noticeable cost to processing time.
retrieveing records from the database, use ($dbocc > 0 )
ReplyDelete