Scanning is slow - should I split?

Almost 4 years ago, I wrote a post entitled Scanning is slow.  Well, to be fair, it is.

The example code that I gave was for counting how many times a substring appeared within a string...

  temp = list

  total = 0

  scan temp,"ABC"

  while ( $result > 0 )

    total = total+1

    temp = temp[$result+3]

    scan temp,"ABC"

  endwhile

I gave a perfectly simply alternative which performed much better in this scenario, using $replace and $itemcount. 

However, I found myself in a similar situation yesterday, but I couldn't use this trick, because I needed to loop through, searching for certain substrings, but without changing the string that I was looping through.

A thought occurred to me, which was that I was always taking the string from after the value I'd found and scanning again, so in this case, what about using $split instead?

To compare against to above, I came up with something like this...

  temp = list

  total = 0

  $result = $split(temp,1,"ABC",lhs,rhs)

  while ( $result > 0 )

    total = total+1

    temp = rhs

    $result = $split(temp,1,"ABC",lhs,rhs)

  endwhile

I tested this by doing 500 iterations, on a string with 500 instances (so total was 250000 at the end)...

$scan: 20.94, 20.67, 20.85 - almost 21 seconds.
$split: 27.09, 27.48, 26.83 - around 27 seconds.

So $split definitely looked worse off, it looked like $scan was better suited to what I needed (as I wasn't interested in the left hand side).

However, these tests have been seeing whether I can use $split to improve my $scan loop, and this seemed a little unfair now I'd proven it couldn't, so I thought I'd flip it on it's head and see if $scan is better at doing what $split is designed for.


So for $split this is quite easy, a simple line which splits the string into two parts...

  $result = $split(temp,1,"~",lhs,rhs)

Using $scan this is a little more complicated, and requires a little string manipulation...

  $result = $scan(temp,"~")

  lhs = temp[1:$result-1]

  rhs = temp[$result+1]

When I compared the two over 2,000,000 iterations, I got these results...

$split: 10.19, 9.78, 10.01 - around 10 seconds.
$scan: 16.60, 16.51, 16.82 - over 16 seconds.

So I've worked out that $split is good for splitting and $scan is good for scanning... Why am I writing this up in a blog post, you might wonder!

Well I then considered the fact that I was splitting a string on a delimiter that I knew existed.  And in the real world situation I was working on, this wasn't a fair assumption, as often the delimiter would not be there.  So I tried this again, but with a different delimiter, that wouldn't be found in the string...

$split: 10.66, 10.73, 10.87 - a little over 10 seconds.
$scan: 9.20, 9.00, 8.44 - around 9 seconds.

So it looks like it's not quite as straightforward as I thought.  However, the saving when it is found is quite a lot larger than the loss when it's not found, so maybe $split is still best overall.

Summary: If you're looking to scan through a string, use $scan, and if you're looking to split a string (especially if you expect the delimiter to be there), use $split.

Training videos for Uniface developers

Yesterday there was a blog post by Ton Blankers on Uniface.info covering HTML5, Javascript and CSS3 Training Videos for Uniface Developers.  

This post highlights a special series of training videos on web topics for Uniface developers. They are available on Uniface.info, and include both introductory and more advanced videos.

They're well worth checking out, as they cover things like Canvas, Geolocation, LocalStorage, JSON and more.

Summary: Check out the full blog post here for more details and links to the videos.

Putlistitems - successive occurrences

I've used putlistitems a lot, it's very useful.  Different switches all it to do different things, so for example, /occ will populate a variable with an associative list of field names and values, something like this...

  putlistitems/occ varMyList,"ENT"

"varMyList" will now be something like "FIELD1=VALUE1·;FIELD2=VALUE2·;etc".

This can further be modified by adding /modonly, which will only add field values to the list which have been modified since the record was retrieved, something like this...

  putlistitems/occ/modonly varMyList,"ENT"

However, the one I tend to use most is /id.  This will take the values of the specified field from multiple occurrences and put them into a list, something like this...

  varMyList = "FIELD1"

  putlistitems/id varMyList

"varMyList" will now be something like "VALUE1·;VALUE1·;etc", repeated for each occurrence.

This allows you to quickly grab the values of the field(s) you're interested in, without having to loop through the occurrences in order to build the list, which is much better for performance.

Or you can also use it to update the representations in a list, for example, backing up a load of field or register values...

  varMyList = "$1·;$2·;$3"

  putlistitems/id varMyList

"varMyList" will now be something like "$1=REG1·;$2=REG2·;$3=REG3".

All of this is well documented in the Uniface manuals, so you may well be asking me why I'm telling you this.  Fair question.

I never realised that the first use of /id, taking the values of the specified field from multiple occurrences, was actually successive occurrence.  This means that it will start on the current occurrence and loop through to the end, completing the hitlist if required.

The manuals make this clear, with the following note...

To ensure that the entire set of occurrences in the component is being addressed, make the first occurrence the current occurrence.

What this means is, use setocc first, like this..

  setocc "ENT",1

  varMyList = "FIELD1"

  putlistitems/id varMyList

I hope I've not been caught out by this in the past!

Summary: putlistitems/id will look at successive occurrences, so if you want the whole hitlist, remember to make sure the current occurrence is the first occurrence.

Uniface Lectures - Deployment

The first "Uniface Lecture" on Modernisation was really interesting, although it did cover a lot of the same ground as the series of blog posts on Modernisation, which I've also posted about before (here, here and here).  It went well though, and I expect the rest in the series to be just as good, if not better!

I did get caught out by the setup required (which we were told about in advance, so it's my own fault), so if you don't already have a browser plugin for "Skype for Business" then you should get that sorted out in advance by following this article.

The next webinar in the series is going to be on Thursday 10th March, 8am and 4pm (CET). 

Deployment

• Standard deployment (with UAR's) versus classic deployment (with URR's and DOL's) 
• The benefits of standard deployment 
• Things to consider when deploying your Uniface application using the 64bit version of Uniface.

Although I'm aware of standard deployment, I've only ever used classic deployment, so I'm interested to find out more about the differences and what I might be missing out on.

Click here for more information and to register

As I said in my last post, I'm just passing this information on because I think they'll be useful.  If you want more information, or to hear it from the horses mouth, check out the website...

http://go.uniface.com/Lectures

Uniface Lectures

Uniface have got a new series of "lectures", or webinars, coming up soon.  These sound like a great idea, and are covering a range of different subjects.

The webinars are one a month, and topics published so far are...

Modernization - Thursday 18th February (8am or 4pm CET)

How to give your Uniface applications a Windows 10 look and feel, and how to work with different screen resolutions.  

Click here for more information and to register

Deployment - Thursday 10th March (TBC)

How to more easily distribute and update your Uniface applications.


JavaScript - Thursday 14th April (TBC)

How to use Uniface in existing web pages.


Mobile - Thursday 12th May (TBC)

How to build your first Uniface mobile app.


Integration - Thursday 16th June (TBC)

How to integrate your Uniface applications with Dropbox and OneDrive. How to use semantic URLs for Uniface RESTful services.

I'm just passing this information on because I think they'll be useful.  If you want more information, or to hear it from the horses mouth, check out the website...

http://go.uniface.com/Lectures

Drag and drop

Yesterday I read an interesting article about drag and drop in Uniface, aptly named Drop It Like It's Hot.  

It actually goes into plenty of detail about how to add drop areas into your own Uniface forms, which is something I'd definitely like to try some time.  But what really interested me was the opening paragraph...

What very little people know is that in the Import Repository Objects component there is a small section where you can drag your export files from Windows Explorer. After you drop them, the File name box is updated with the list of files that you dropped. The red section is where the drag-and-drop field is located.

I tried this today and it definitely works.  You learn something new every day!

Summary: You can drag and drop export files onto the Import Repository Objects popup, which is really quick and convenient.

Modernising Uniface - part 3

Over the past couple of weeks I have posted about Modernising Uniface (and here), and this is the third and final post in this series.

The third post from Theo Neeskens was primarily concerning the new flat buttons, which look like this...

As you can see, the new flat blue buttons, on a white background, are on top of the old grey/grey.

The first thing they needed to do was categorise the different painted button types, and they came up with five...

1. IDFButtonBottom - for the big buttons at the bottom of dialogs
2. IDFButtonSide - for the big buttons at the right-hand side of dialogs
3. IDFButtonSpecial - for the other buttons
4. IDFButtonImage - for the small buttons without labels (eg. ">>")
5. IDFButtonHeader - for the buttons which make up the header on simulated grids

So then there are a few jobs that needed to be done...

• Change the properties in the configuration file for these widget types
• Match modelled widgets with painted widgets
• Set the properties on the painted buttons
• Set the properties on the modelled buttons

Again, many of the these jobs were achieved with tools, and all of these have been provided in a zip file, to be downloaded and modified if you wish, but not supported by Uniface and used at your own risk.

The full blog post is definitely worth checking out, including the "Wishes" section at the end... Modernizing Uniface 9.7: The buttons

Summary: Uniface has successfully (in my opinion!) modernised it's IDE in version 9.7, meaning we don't have to wait for Uniface 10.

Modernising Uniface - part 2

I posted last week about Modernising Uniface, which included my comments about Theo Neeskens post on UnifaceInfo.com.  I received some comments that I was talking purely about the IDE, and not the language itself.  So let me start by stating that this post of about the modernisation of the IDE, and not the language itself.

This time the post is all about the effort it took to make the seemingly simple move from grey backgrounds (which psychologically make software feel outdated) white clean, clear, white backgrounds, like this...

For both of these dialogs, the older grey version sits behind the newer white version.  You can also see the buttons have changed, but we've been promised more about those in the next blog post.

The main problems they hand with this change included...

• Forms having index, foreground or background colour set
• Painted entities having index, foreground or background colour set (and being painted at least 3 characters wide)
• Entities having foreground or background colour set in the model
• Grids not having borders
• Multi-occurrence entities not having borders

A couple of these have been resolved by making new properties available, but all of them required fix programs to find the affected items and update them to the new properties.  All of the tools have been provided in a zip file, to be downloaded and modified if you wish, but not supported by Uniface and used at your own risk.

I've had my hands on Uniface 9.7 now (which I hadn't when I wrote the first post) and I can say that it definitely feels more up-to-date, and I agree that the white backgrounds and blue buttons are a large part of that.  We've actually been changing our web offering to a similar colour scheme over the past few months, so I can't disagree with their choice!

Again, the full blog post is definitely worth checking out... Modernizing Uniface 9.7, Part 2.

Summary: Uniface is modernising it's IDE, and we're not going to have to wait for Uniface 10 to get in on the action.

Modernising Uniface

Uniface have been putting a lot of work into the new Uniface 10 IDE, and this is something I'm looking forwards to getting my hands on, but it could be a while yet.  Good things come to those who wait, they say.

Well the good news is that the delays in Uniface 10 have led to the decision to release a Uniface 9.7.  Whilst they may be planning to put limited time into this, whilst focusing their efforts on Uniface 10, it looks like they have big plans to modernise.

In the first of a 10 part series, Theo Neeskens has outlined the objectives and the first set of changes - the start screen.

The objectives were straightforward enough...

1. New start page - Windows 10-esque - big change.
2. Change grey backgrounds to white backgrounds.
3. Use trendy flat buttons.
4. Fresh look and new colour scheme - something different.

Looks like they met the objectives, based on these screenshots...

Now that's a radical change!  I think changes like this will be very well received, and help stop people getting such a negative first impression of Uniface as a language.

The full blog post is definitely worth checking out... Modernizing Uniface 9.7 in 10 easy steps.

Summary: Uniface is modernising, and we're not going to have to wait for Uniface 10 to get in on the action.

Where to put your code

There's an interesting blog post by Dennis Vorst gone up on the Uniface.info blog, titled Where to put your code.  As the title suggests, it's about which places are best for different types of code, based on their content.  

There are a number of options; conceptual at the entity or field level, locally at the entity or field level, within the component, a local procedure or a global procedure.  All of these have their merits.  

I've posted my response, defending the mighty global procedure, but why don't you have a read and add your opinion in the comments?

Summary: Check out this blog post... http://unifaceinfo.com/where-to-put-your-code

Performance of getitem

I've previously posted about how scanning is slow, and I stand by that.  But I've recently discovered that in some situations, it can be less slow than using getitem.  

I was doing a code review of some old code and found that it was using $scan in a situation where I thought you'd usually use getitem, and wondered why someone would have done it this way.  But before I replaced it, I wanted to check the performance to see what difference I would be making by changing it, and I was surprised by the results!

The code was designed to check if a variable matched one of a reasonably large number of items.  For this example, if stuck with 10 items...

  if ( temp = "ONE" | temp = "TWO" | temp = "THREE" | temp = "FOUR" | temp = "FIVE" | temp = "SIX" | temp = "SEVEN" | temp = "EIGHT" | temp = "NINE" | temp = "TEN" )

    ;testing condition

  endif

I could have used a line continuation marker, but you get the idea.

What the developer had done is replaced this set of conditions with a single $scan, like this...

  list = "|ONE|TWO|THREE|FOUR|FIVE|SIX|SEVEN|EIGHT|NINE|TEN|"
  if ( $scan(list,"|%%temp%%%|") > 0 )

    ;testing condition

  endif

Note that this is not a Uniface list, a "bar" or "pipe" character has been used as the delimiter - this is placed at the beginning and end of the value to ensure it is not found as a sub-part of another longer value.  It This takes up a lot less space, and is perfectly readable, but I was concerned about performance.  

To test this, I wanted to make sure it was fair, so I decided to always check both the first and the last item in the list in each iteration.  I also wanted to test in a few different ways, and I came up with 4...

1) Set of conditions

  temp = "ONE"

  if ( temp = "ONE" | temp = "TWO" | temp = "THREE" | temp = "FOUR" | temp = "FIVE" | temp = "SIX" | temp = "SEVEN" | temp = "EIGHT" | temp = "NINE" | temp = "TEN" )

    ;testing condition

  endif

  temp = "TEN"

  if ( temp = "ONE" | temp = "TWO" | temp = "THREE" | temp = "FOUR" | temp = "FIVE" | temp = "SIX" | temp = "SEVEN" | temp = "EIGHT" | temp = "NINE" | temp = "TEN" )

    ;testing condition

  endif

2) $scan a bar delimited string

  list = "|ONE|TWO|THREE|FOUR|FIVE|SIX|SEVEN|EIGHT|NINE|TEN|"
  if ( $scan(list,"|ONE|") > 0 )

    ;testing condition

  endif
  list = "|ONE|TWO|THREE|FOUR|FIVE|SIX|SEVEN|EIGHT|NINE|TEN|"

  if ( $scan(list,"|TEN|") > 0 )

    ;testing condition

  endif

3) getitem/id a Uniface list

  list = "ONE·;TWO·;THREE·;FOUR·;FIVE·;SIX·;SEVEN·;EIGHT·;NINE·;TEN"

  getitem/id temp,list,"ONE"

  if ( $status > 0 )

    ;testing condition

  endif

  list = "ONE·;TWO·;THREE·;FOUR·;FIVE·;SIX·;SEVEN·;EIGHT·;NINE·;TEN"

  getitem/id temp,list,"TEN"

  if ( $status > 0 )

    ;testing condition

  endif

4) $item a Uniface list

  list = "ONE·;TWO·;THREE·;FOUR·;FIVE·;SIX·;SEVEN·;EIGHT·;NINE·;TEN"

  if ( $item("ONE",list) != "" )

    ;testing condition

  endif

  list = "ONE·;TWO·;THREE·;FOUR·;FIVE·;SIX·;SEVEN·;EIGHT·;NINE·;TEN"

  if ( $item("TEN",list) != "" )

    ;testing condition

  endif

I wasn't really sure what I was expecting, but I thought the $scan would be the worst performance.  I tested over 2,000,000 iterations, and here's what I got...

1) Set of conditions: 37.30, 36.95, 37.83 = 37.36 secs
2) $scan a bar delimited string: 21.76, 21.97, 21.24 = 21.66 secs
3) getitem/id a Uniface list: 24.46, 24.22, 24.89 = 24.52 secs
4) $item a Uniface list: 22.65, 23.25, 23.47 = 23.12 secs

So the slowest was the set of conditions.  I didn't test it, but knowing that if statements shortcut I figure that if the item was always passing the first condition it would be quick, but because half of my test items were only passing the last condition, it would have to check each of the conditions in the set before it passed.

Although there wasn't a big difference, what surprised me is that the getitem/id and $item were actually slower than the $scan in this case.  I then remembered back to a conversation on the Uniface-L mailing list, which talked about how Uniface handles lists in the background.  The description there indicates that an array is built in the background, which means there is an upfront cost for calling getitem/id (or $item) once, but then if you're looping through it is much quicker to access the rest, because the array can be used.  However, because I'm rebuilding the list each time, that means the array needs to be rebuilt each time.  

This means that actually $scan can be used to improve the performance when checking that an item is in a list, as long as you're only checking this list once and it's not going to be re-used.  I expect there to be a point at which the number of times the list is re-used means that using getitem/id (or $item) would become better for performance.

Also, I've not tested different lengths of lists.  However, given the results and my reasoning for why the results ended up this way, I would have thought extending the list would simply emphasize the results.

Summary:  Checking if an item is in a list can be done a number of ways, and if it's being done a lot of times, performance can be eeked out by using a $scan, surprisingly!

Modernising the client application - icons in the menu

A lot of the Uniface development that I do these days is actually web development, which means I spend a lot of time with Javascript and JQuery, and less time with Uniface procscript.  However, we still have client applications as well, and they can look tired and dated, so anything we can do to make them look more modern, can be a big win!

This is a simple little trick that appeared in Uniface 9.6.  It's actually been around for a while in dropdown menus, first as an undocumented feature, and then documented in Uniface 9.5 - I wrote a blog post on this almost three years ago.  However, the same trick now works in menus.

You can reference images in the menu items like this...

You might not be able to read the image clearly, so the three menu items are...

1. ^U_REM_SELECT·!Cu%t
2. ^U_SAVE·!Copy
3. ^U_INS_SELECT·!Paste

In this case, the image is at the start, with a gold-not (or gold-exclamation-mark) as a delimiter, and the menu item text after that.

This appears in the client application this this...

As you can see - lovely icons!  Please note that you would need these glyphs in your system for it to work, these are not provided as part of Uniface itself.  

You can include icons in one of two ways...

• ^glyph - glyphs need to be compiled into UOBJ (always available and performs well).
• @filename - images loaded from a fileserver (not always available and performs badly).

I would recommend the glyph option though, for performance and reliability.

You may also note that the percent character (%) can be used to denote which character should work when traversing the menu using the "Alt" key.  If you do not explicitly denote this, the first character will be used.

This is documented in the "Defining and Using Menus" section of the manuals, but can be hard to find unless you know what you're looking for!  It's worth reading this for more information though.

Summary:  It's possible to make the client application look a little more modern now by including icons in the menu bars.

Undocumented feature - current assignment file

One great feature of Uniface is it's great flexibility and configurability, through the use of different assignment (.asn) files.  These are used to define paths to the different runtime files and server locations, etc.  The problem can then come though, especially with multiple environments, that you don't know which assignment file you're using!

I wrote a post a while ago about the undocumented $assignments function, which allows you to get access to the assignment file settings that you are currently using, but this doesn't help you get to the filename itself.

One suggestion I've seen before is to add a logical to each assignment file which specifies the name/location of the file.  However, it's way too easy for someone to move, copy or rename the file and neglect to update the logical, and then worse than not knowing, you think you do know but the information you're being given is inaccurate.

Luckily Uniface 9.6.06 has solved this problem with a new undocumented feature of $processinfo...

  putmess $processinfo("cmdline")

This will put the command line used to access the application into the message buffer.  For example...

  "\\server\location\uniface\bin\uniface.exe" /asn=\\server\location\adm\uclidev.asn /ini=\\server\location\ini\me.ini APPSHELL

As you can see, the assignment file and ini file are both specified in the command line, along with the application shell as well.  This should give you all the information you need, although you'll need to do some string manipulation to pull out the bit(s) you want.

This was recently discussed on the UnifaceInfo.com forums, so you can check there for more context, and sign up if you haven't already!

Summary: You can access the current assignment file location, but accessing the command line, as long as you're happy to use an undocumented/unsupported feature.

Indirection with dollar registers

Indirection in Uniface is a wonderful thing.  You can populate a local/component/global variable with a field name, and then use indirection to get the field value of the field that the variable references.  You can also use it to set the referenced field to a value, like this...

  fld1 = "fld2"

  @fld1 = "Test"

  if ( fld2 = "Test" )

    ;this will be true

  endif

In this example, using the "@" character to indicate indirection, it's the field which is referenced by "fld1" and not "fld1" itself which is updated.

You can even do recursive indirection, which is very cool!  Not something I've had a need for myself, but very clever all the same.

However, today I was trying to populate a register the same way, using code something like this...

  reg50 = "$50"

  @reg50 = "Test"

  if ( $50 = "Test" )

    ;this will be false!

  endif

In this example, the register was still blank (or whatever the register was before hitting this code).

To be fair, this is clearly stated in the Uniface manuals...

Only fields can be referenced indirectly; the name reached by the indirect reference cannot be a variable. The following example is incorrect because the indirection refers to a general variable rather than to a field:

$1 = "$99"

@$1 = "This string does not go to $99."

But I still wanted to be able to set registers in an indirect way, so I came up with this alternative...

  putitem/id list,"$50","Test"

  getlistitems/id list

  if ( $50 = "Test" )

    ;this will be true

  endif

This works by populating a list with the register and the value, and then uses getlistitems to populate them.  The "/id" mode copies items from the associative list into one or more fields, or registers in this case.

For example, you could use a simple for loop to clear out all of the dollar registers...

  for count = 1 to 99

    putitem/id list,"$%%count%%%",""

  endfor

  getlistitems/id list

This would set each of the 99 dollar registers to blank.

Summary: Indirection works great for fields, but not for registers - however, you can use getlistitems to achieve the same outcome.