Putlistitems - successive occurrences

I've used putlistitems a lot, it's very useful.  Different switches all it to do different things, so for example, /occ will populate a variable with an associative list of field names and values, something like this...

  putlistitems/occ varMyList,"ENT"

"varMyList" will now be something like "FIELD1=VALUE1·;FIELD2=VALUE2·;etc".

This can further be modified by adding /modonly, which will only add field values to the list which have been modified since the record was retrieved, something like this...

  putlistitems/occ/modonly varMyList,"ENT"

However, the one I tend to use most is /id.  This will take the values of the specified field from multiple occurrences and put them into a list, something like this...

  varMyList = "FIELD1"

  putlistitems/id varMyList

"varMyList" will now be something like "VALUE1·;VALUE1·;etc", repeated for each occurrence.

This allows you to quickly grab the values of the field(s) you're interested in, without having to loop through the occurrences in order to build the list, which is much better for performance.

Or you can also use it to update the representations in a list, for example, backing up a load of field or register values...

  varMyList = "$1·;$2·;$3"

  putlistitems/id varMyList

"varMyList" will now be something like "$1=REG1·;$2=REG2·;$3=REG3".

All of this is well documented in the Uniface manuals, so you may well be asking me why I'm telling you this.  Fair question.

I never realised that the first use of /id, taking the values of the specified field from multiple occurrences, was actually successive occurrence.  This means that it will start on the current occurrence and loop through to the end, completing the hitlist if required.

The manuals make this clear, with the following note...

To ensure that the entire set of occurrences in the component is being addressed, make the first occurrence the current occurrence.

What this means is, use setocc first, like this..

  setocc "ENT",1

  varMyList = "FIELD1"

  putlistitems/id varMyList

I hope I've not been caught out by this in the past!

Summary: putlistitems/id will look at successive occurrences, so if you want the whole hitlist, remember to make sure the current occurrence is the first occurrence.

Modernising Uniface - part 2

I posted last week about Modernising Uniface, which included my comments about Theo Neeskens post on UnifaceInfo.com.  I received some comments that I was talking purely about the IDE, and not the language itself.  So let me start by stating that this post of about the modernisation of the IDE, and not the language itself.

This time the post is all about the effort it took to make the seemingly simple move from grey backgrounds (which psychologically make software feel outdated) white clean, clear, white backgrounds, like this...

For both of these dialogs, the older grey version sits behind the newer white version.  You can also see the buttons have changed, but we've been promised more about those in the next blog post.

The main problems they hand with this change included...

• Forms having index, foreground or background colour set
• Painted entities having index, foreground or background colour set (and being painted at least 3 characters wide)
• Entities having foreground or background colour set in the model
• Grids not having borders
• Multi-occurrence entities not having borders

A couple of these have been resolved by making new properties available, but all of them required fix programs to find the affected items and update them to the new properties.  All of the tools have been provided in a zip file, to be downloaded and modified if you wish, but not supported by Uniface and used at your own risk.

I've had my hands on Uniface 9.7 now (which I hadn't when I wrote the first post) and I can say that it definitely feels more up-to-date, and I agree that the white backgrounds and blue buttons are a large part of that.  We've actually been changing our web offering to a similar colour scheme over the past few months, so I can't disagree with their choice!

Again, the full blog post is definitely worth checking out... Modernizing Uniface 9.7, Part 2.

Summary: Uniface is modernising it's IDE, and we're not going to have to wait for Uniface 10 to get in on the action.

What are handles and should I be using them?

The Uniface manuals describes a handle as...

A handle is a reference to an object, such as a component instance, entity, occurrence, or field. Handles can be used to call operations on the referenced object, or to pass these objects as variables and parameters.

If you're used to pointers, you may at first glance think that these are the same, but they're not.  A pointer contains the address of an item in memory and is therefore fixed, whereas a handle is an abstract reference, which means the address can be changed without breaking the reference.  This makes them very useful.

In Uniface you can have "public" or "partner" handles, which allow you to define which types of operations can access them.  They are also datatypes available for local, component and global variables, depending on the scope required.  I'm not going to discuss either of these points in detail, but I think it's handy to know.

As the description in the manual states, you can create a handle that references most Uniface objects, and there are different methods for creating these handles:  Component instances ($instancehandle), entity collections ($colhandle), entity occurrences ($occhandle), fields ($fieldhandle) and also OCX controls ($ocxhandle).  

I'm going to focus on component instances.  More specifically, I'm going to focus on creating new component instances, using newinstance and handles.  

As you might expect, creating a new instance of a form has a little lead time.  Using a handle doesn't stop this lead time, but what it does do, is mean that it only has to happen once.  This means that using a handle for a single form activation is not going to give you any advantage (unless you just love handles!) but when calling operations on the same form multiple times, handles could give you a performance improvement.

This is something like how you could activate a form...

  activate "TEST_FORM".test_oper("Test")

In fact this does an implicit newinstance as well (if it can't already find an instance with the same name as the form) and so explicitly your code could look like this...

  newinstance "TEST_FORM","TEST_FORM"

  activate "TEST_FORM".test_oper("Test")

If you were to replace this with a handle then you may use something like this...

  newinstance "TEST_FORM",theHandle

  theHandle->test_oper("Test")

Notice that in this case we have used a handle instead of an instance name in the newinstance call, and then we can use this handle in place of the activate command.  The -> is referred to by Uniface as the "dereference" operator, and is processed with a precedence of 2 (with only field indirection being higher).

So let's try these two different ways of activating an operation (in this case a very simple operation that simply returns out immediately).  The newinstance command is done before the loop, and the activate (or equivalent handle call) is done inside the loop, 200,000 times...

• Using activate: 2.56, 2.61, 2.60 (about 2.6 seconds)
• Using a handle: 2.39, 2.35, 2.39 (about 2.4 seconds)

As you can see, using a handle is marginally quicker, although you'd have to be making a lot of calls before you really noticed the difference.  

I thought that this would be even more useful when the handle references a component instance that is running on another server, such as when a form is mapped to run elsewhere, as the I thought the initial creation of the new instance would take longer here.  So I ran the same operation 20,000 times (after mapping the form)...

• Using activate: 13.39, 14.35, 12.99 (about 13.5 seconds)
• Using a handle: 13.56, 13.12, 12.91 (about 13.0 seconds)

Not much difference here either.  Having said that, maybe on a chatty network the difference would be greater.

Please note:  You must return out of the operation that you call, if you exit out then this will destroy the handle and future calls to the operation will fail with a $status of -1.  To clean up a handle you can either set it to 0 or "" (empty string). 


Summary: Handles are abstract references to pretty much any Uniface object.  They can be useful, especially if you want to call the same object multiple times, but they don't seem to have the performance benefits that I was expecting, not when it comes to component instances anyway.

Reducing your memory footprint

One common use for the while loop is to loop through a set of retrieved records.  This can also be done with a for loop, as I discussed in a previous post about types of for loops.  The example I gave was something like this...

  setocc "ent",1

  while ( $status > 0 )

    ;do something

    setocc "ent",$curocc(ent)+1

  endwhile 

This is nice and simple, and also pretty efficient.  There is one big downfall with it though, and that's memory.  In order to improve performance when you do a retrieve, Uniface works out what the "hitlist" is, but doesn't actually pull all of the retrieved records into memory.  As you loop through, more and more of the hitlist is completed, and the records are pulled into memory.  

To demonstrate this, I retrieved 600 records and then looped through them, using Process Manager to track the memory of the "uniface.exe" program...

• Initial value: 14.56Mb
• 600 records retrieved: 14.64Mb (+0.08Mb)
• Looped to occurrence 100: 14.91Mb (+0.27Mb)
• Looped to occurrence 200: 15.30Mb (+0.39Mb)
• Looped to occurrence 300: 15.70Mb (+0.40Mb)
• Looped to occurrence 400: 16.10Mb (+0.40Mb)
• Looped to occurrence 500: 16.54Mb (+0.44Mb)
• Looped to occurrence 600: 16.93Mb (+0.39Mb)

As you can see, retrieving the hitlist is just the first step, and a small one at that, the data is not read in until you loop through the occurrences.

If you had retrieved a rather large amount of data, looping through it like this would continue to hold all of that data in memory.  In the situation where you're processing records one by one, you often no longer need the record after it's been processed, so it's a good idea to discard the record, to free up the memory.  

Each time you discard a record, it removes it from the hitlist and then does an implicit setocc to the next record, which becomes the new current record.  Another thing that discard does is set $status to be the occurrence number, or 0 if there are no records left in the hitlist, just like setocc does.  This allows us to build a very similar loop to the one we had before...

  setocc "ent",1

  while ( $status > 0 )

    ;do something

    discard "ent",$curocc(ent)

  endwhile

An alternative that I often see is something more like this...

  setocc "ent",1

  while ( $hits("ent") > 0 )

    ;do something

    discard "ent",1

  endwhile

I'm not a fan of this one though, as $hits can sometimes have performance problems of it's own, caused by it's tendency to complete the hitlist in order to return the count.  

So, let's check how these different methods stack up against each other performance-wise, using the same 600 records as before...

• Original loop: 6.37, 6.02, 5.90 (around 6 seconds)
• Discard loop with $curocc: 5.95, 5.95, 5.86 (just under 6 seconds)
• Discard loop with $hits: 5.95, 6.00, 5.89 (just under 6 seconds)

As you can see, there isn't that much difference in the approaches as far as performance goes.  In this case $hits hasn't caused a problem, but if I remember correctly, it's environment and/or database specific, so that's probably why.  

Summary: Using discard with a large record set is a good idea, because it reduces the memory footprint of the userver, and at no noticeable cost to processing time.  

Variables as lists

I think you'd be hard pushed to find a Uniface developer that didn't know that a string variable can be used to hold a list of values.  The Uniface list construct uses a gold-semi-colon as the list delimiter, denoted in the manuals as ; but in code looks like ·; - for example...

  s = "27/Sep/2012·;28/Sep/2012·;29/Sep/2012"
      ;27/Sep/2012·;28/Sep/2012·;29/Sep/2012

I spent quite a long time today trying to work out why such a simple thing was not working for me.  It took quite a long time for me to realise that I'd accidentally declared (or rather, re-used) a numeric variable.  This gives a rather different result...

  n = "27/Sep/2012·;28/Sep/2012·;29/Sep/2012"

      ;27·;28·;29

I wasn't previous aware that a numeric variable could hold a list in this way, but it's handy to know! 

The same also works with dates...

  d = "27/Sep/2012·;28/Sep/2012·;29/Sep/2012"

      ;2012092700000000·;2012092800000000·;2012092900000000

Take a look at this screenshot to see the results...

I'm not going to go through all the data types to confirm which ones it works with, but I suspect it would work with all of the simple ones (such as datetime and float) but not with the more complex ones (such as handle and occurrence).

Summary: It's not just string variables that can be used to hold lists; numeric and date variables also can, along with many others I suspect.

Types of for loops - part two

In part one I talked about the forlist statements and how these could be used to iterate through Uniface lists for easily.  The next thing I want to talk about is looping through entity occurrences. Personally I've always done this using a combination of setocc and $curocc, something like this...

  setocc "ent",1

  while ( $status > 0 )

    ;do something

    setocc "ent",$curocc(ent)+1

  endwhile

This has the advantage of not needing to use any variables for the loop.  However, it may be better for performance if a similar loop was used, but using variables to control it...

  count = 0

  stat = $hits(ent)

  while ( count < stat )

    count = count+1

    setocc "ent",count

    ;do something

  endwhile

Another alternative would be to use the new statement forentity, which was also added in Uniface 9.5...

  forentity "ent"

    ;do something

  endfor

As you can see, the code is much more concise.  There is no need to initialise the count variable or  $status, everything is done as part of the forlist statement, and the incrementing and extracting are done automatically.  The "count" variable is optional, if you don't need it then you don't need to include it.

So let's test these three blocks of code over 65,000 iterations...

• while ($curocc) = 00:00.47, 00:00.47, 00:00.47 (about half a second)
• while (variables) = 00:04.62, 00:04.25, 00:05.10 (about 5 seconds)
• forentity = 00:00.37, 00:00.34, 00:00.35 (about a third of a second)

As you can see, the new forentity is more concise code and also performs better, fairly significantly over some alternatives.

I was surprised by how slow the second method was compared to the first.  The only explanation I have for this is that $hits is taking a long time to complete the hit-list before the loop starts, rather than completing the hit-list as it goes through the loop.  Turns out I’ve been doing it a pretty efficient way all along, but I like the simplicity of the new forentity statement.  Which leads me to an almost identical summary as in part one. 

Summary: Whilst I have previously always used while loops, I shall now be considering switching the forentity loops, for iterating through entity occurrences.