Tuesday, 25 November 2014

Syntax strings in different modes

Earlier I wrote a post about syntax strings versus literal strings, which involved me diving into the Uniface manuals to check my facts.  I talked about using $syntax to convert a literal string into a syntax string - very useful!  

However, what I didn't know before was that you can have different modes.  This is something that appears to have been added in Uniface 9.4.01, without me noticing.

There's a lovely table in the Uniface manuals to describe them, but I've tested their examples and I think there are some inaccuracies.  So I'm going to try and lay them out for you here...


This is the default behaviour, but is also represented by '%[X]'.  In this case, the usual pattern matching rules apply, with syntax codes being used as wildcards to represent patterns of characters.

  • Proc code: $syntax("D&G")
  • Syntax string: '%[X]%D&G'
  • Matches: "DOG", "DIG", "DUG", etc.

You can see the syntax string starts with the mode defined in this case, which is optional for classic mode, because this is the default.

Case Sensitive

In this mode, all characters will only match with characters of the same case.  Also, syntax codes will be treated as their literal characters, and not as wildcards.

  • Proc code: $syntax("D&G","CS")
  • Resulting syntax string: '%[CS]D%&G%[X]'
  • Matches: "D&G"

You can see the syntax string starts with the mode defined in this case, but it also ends with the exit mode, switching back to the default/classic mode.

The manuals suggest that in the proc code the mode is passed in without the double quotes, but I found this had to be a string for it to work.  They also suggest that the mode "S" can be used instead of "CS" for the same thing - I found that only "CS" works in proc code, but you can use '%[CS]' or '%[S]' in syntax strings you write yourself.

Case Insensitive

In this mode, all characters will match with characters of either upper or lower case.  Again, syntax codes will be treated as their literal characters, and not as wildcards.

  • Proc code: $syntax("D&G","CI")
  • Resulting syntax string: '%[CI]D%&G%[X]'
  • Matches: "D&G", "d&g", "D&g" and "d&G"

Again, I had to use a string to define the mode in the proc code, and only "CI" worked, but '%[CI]' and '%[I]' both worked in syntax strings I wrote myself.

NLS Locale

In this mode, all characters will match with characters of either upper or lower case, depending on the National Language Support (NLS) locale that the system is in (can be checked or set using $nlslocale).  Again, syntax codes will be treated as their literal characters, and not as wildcards.

  • Proc code: $syntax("i#B","NLS")
  • Resulting syntax string: '%[NLS]i%#B%[X]'
  • Matches: If your local is Turkish (tr_TR) then "i#B" and "İ#b", but not "I#B"

Once more, I had to use a string to define the mode in the proc code, and only "NLS" worked, but '%[NLS]' and '%[N]' both worked in syntax strings I wrote myself.

Mixing modes

You can also define a combination of modes in a single pattern, something like this...
  • Proc code: $syntax("%[CI]D%&%[CS]G%[X]")
  • Resulting syntax string: '%[CI]D%&%[CS]G%[X]'
  • Matches: "D&G" and "d&G"

There's a slight typo in the manuals with the proc code here, as they have missed the "%" from the beginning!  

Summary: You can use different modes in syntax strings, either specifying the case-sensitivity or relying on the NLS locale, but be careful if you're using $syntax because you'll lose the wildcards.

Monday, 24 November 2014

Syntax strings versus literal strings

There are a number of very important differences between syntax strings and literal strings, which I will attempt to highlight for you here.

Firstly, the Uniface definition of a syntax string...
Uniface enables you to determine if the data in a string value matches a desired pattern using syntax strings.  A syntax string is a group of characters and syntax codes enclosed in single quotation marks (').
So that's the first big difference right there, literal strings are in double quotes (") and syntax strings are in single quotes (')...

  literalString = "Literal string"
  syntaxString = 'Syntax string'

A syntax string is used for pattern matching, but they are much more simplistic than the regular expressions that you might be used to in other languages.  The syntax codes are quite straight forward...

  • # - one digit (0-9)
  • & - one letter (A-Z, a-z)
  • @ - one letter, digit or underscore (A-Z, a-z, 0-9, _)
  • ~& - one extended letter
  • ~@ - one extended letter, digit or underscore
  • ? - one ASCII character
  • A-Z - that letter, in uppercase
  • a-z - that letter, in uppercase or lowercase

On top of these are a few other syntax codes of note...

  • If you want to search for the literal version of a syntax code, eg. you want to search for the hash character (#) not a digit (0-9), then you can escape the syntax code using % before it.  Therefore, to search for a hash character it would be '%#' and to search for a percentage character it would be '%%'.
  • If you want to search for a unknown number of syntax code, eg. you want to search for 2 or more digits, then you can use * after it.  Therefore, to search for 2 or more digits it would be '###*' - in this case the first two hash characters represent one digit each, and the third hash character is part of the the syntax code '#*', which means zero or more (0-n) digits.
  • If you want part of the pattern to be optional (to match the pattern or blank) then you can put rounded brackets around it, using ( and ).  For example, '##(#)' would match with 2 or 3 digits (but not more than 3).
  • Any other character is treated literally as that character.  

Uniface also gives a handy function that can be used to convert a literal string into a syntax string, sensibly named $syntax...

  syntaxString = $syntax("Literal string")

This can be especially useful if you're storing the pattern in the database, or some other configuration.

Here are a few examples from the Uniface manuals...

Proc with Syntax String
if ('#' = "123")
if ('#*' = "123")
if ('#*' = vValue)
if ('#*' = "%%vValue")
if ('&###' = "1234")
if ('@###' = "1234")
if ('?' = "A")
if ('??' = "A")
if ('??*' = "A")
if ('?' = "ABC")
if ('??*' = "ABC")
if ('(#(-))&&&' = "ABC")
if ('(#(-))&&&' = "1ABC")
if ('(#(-))&&&' = "1-ABC")
if ('(#(-))&&&' = "12ABC")

I've had (occasionally heated!) discussions with developers before, when they have said that...

  if ( myVar = "Y" )


  if ( myVar = 'Y' )

...are interchangable.  

Yes, I can see that the pattern 'Y' only matches with the string "Y" and nothing else, but they are entirely different ideas, and should not be used interchangeably.  If you want to only match with "Y", then use the literal string "Y", because that's what you mean.  

In my experience, developers who say that these are interchangable are developers who don't understand what the differences are.

Summary: Syntax strings are very useful for pattern matching, but very different than literal strings.  Know the difference, and know when to use which.

Thursday, 23 October 2014

Fetching web content

I've been working on fetching web content recently, mostly for building authentication workflows, using the likes of Facebook and Google to act as authentication services.  I'm going to go into detail about how I've done those, not in this post at least, but more simply how I made the requests from my Uniface service form, out to those external services. 

There are a number of different HTTP methods.  Generally speaking, when you are retrieving data only then the method is "GET", whereas if you are sending data that may perform an action as well then the method is "POST".  There are others, but I'm going to stick with these two for now, as they are more commonly used.

In fact, I'm going to start with "GET" first.  

The first thing you need to do is consider the character set.  Your system may be set in different character sets, but you need to make sure the character set of your request matches the service you are calling, in my case it was UTF-8...

  vCharSet = $sys_charset ;backup current setting for later
  $sys_charset "UTF8"   ;set to the character set we need

We then want to create a new instance of the "UHTTP" component - this is the Uniface component that is going to do most of the hard work for us...

  newinstance "UHTTP",vHandle
  if $status < 0 & $procerror < 0 )
    $sys_charset = vCharSet ;restore character set
    return -101             ;error handling!

Once you've got your instance, the next step is to define how the component responds to mismatched or expired certificates, and how it calculates the content length.  By default it will error if there is a certificate error and you have to manually calculate the content length yourself.  It's a binary switch, so to switch them all off we do the following...

  activate vHandle.SET_FLAGS(7)
  if $status < 0 & $procerror < 0 )
    $sys_charset = vCharSet ;restore character set
    return -102             ;error handling!

We're now ready to make the request.  In this example, I'm just going to grab the Google homepage...

  activate vHandle.SEND("http://www.google.co.uk","GET","","","",vContent,vResponse)
  if $status < 0 & $procerror < 0 )
    $sys_charset = vCharSet ;restore character set
    return -103             ;error handling!

The parameters are:

  1. URL (string : in) - the URL you're sending the request to.
  2. Method (string : in) - the method being used (this is not checked by "UHTTP").
  3. Username (string : in) - if you're using a secure URL, you may need to populate this.
  4. Password (string : in) - again, you may need to populate this.
  5. Headers (string : inout) - the HTTP request headers in and the response headers out.
  6. Content (string : inout) - in the case of a "GET" this is out only and the page contents.
  7. Response (string : out) - the HTTP response headers.
The $status should be set to 200 for a successful response (to match the HTTP status code for success) or it may be set to 1.  If it is set to 1 then this means that the content was larger than the parameter limit (10Mb) and therefore you will need to get the rest of the content from the buffer, like this...

  while $status = 1 )
    activate vHandle.READ_CONTENT(vExtra)
    if $status < 0 & $procerror < 0 )
      $sys_charset = vCharSet ;restore character set
      return -104             ;error handling!
    vContent = "%%vContent%%vExtra%%%"

You now have the full page contents, but don't forget to restore the character set...

  $sys_charset = vCharSet ;restore character set

Doing a "POST" is very similar, except for a couple of differences:
  1. To replicate a web browser "POST", which you are usually doing, you need to make sure the 5th parameter of the SEND call is populated with the following header.. "Content-Type=application/x-www-form-urlencoded"
  2. The 6th parameter of the SEND call needs to be populated with the data that you are sending in your request.  If this is larger than 10Mb then you will need to have a WRITE_CONTENT loop (similar to the READ_CONTENT loop) before the SEND, in order to populate the buffer with the full contents.
This all works rather well.  However, I've found two major limitations along the way.

Firstly, it was not possible to send a request to a URL which had a colon (:) character after the protocol.  For example, when trying to get a user's LinkedIn ID you would use the following URL...


The colon (:) character after "/people/~" broke the URL.  Luckily I'm using the past tense here, this was fixed in patch X504 (for Uniface 9.6.05).  This works fine now we've installed the patch.

Secondly, the observant of you may have noticed the 6th parameter of the SEND call is defined as a string.  This is great for grabbing HTML page source from a website, or even doing most webservice calls, as these tend to return either XML and JSON strings of data.  However, it makes creating something like a Dropbox interface impossible, or at least very limited.  You can grab text files, but nothing else!  No images, no Office documents, no PDFs.

Unfortunately there's no word from Uniface on when this second issue might be resolved.

Summary: It's possible to use the "UHTTP" component to request text data, either from a web page or a webservice.  Just don't expect it to work with binary files, yet!

Tuesday, 30 September 2014

Rosetta Code

Recently I was pointed an interesting website, or at least, I thought it was interesting.  And that's Rosetta Code.  I'll let them describe what it's all about...

Rosetta Code is a programming chrestomathy site. The idea is to present solutions to the same task in as many different languages as possible, to demonstrate how languages are similar and different, and to aid a person with a grounding in one approach to a problem in learning another. 

Surprise, surprise; Uniface was nowhere to be seen.  So I've rectified that problem, and added Uniface to the list of programming languages.

I didn't have a lot of time on my hands, so I cheated and copied the contents from Wikipedia, in order to create the base page.  Some of the content is out of date, as it still lists Uniface as a Compuware product, but it's a start.

The idea now is to go through and complete some of the tasks not yet implemented in Uniface, which is currently all of them!  I plan to devote a little bit of time to this myself, but I guess this is a bit of a call out to Uniface developers - go forth and complete tasks!

Summary: Rosetta Code looks like an interesting site for comparing different languages, but Uniface examples are non-existent.

Wednesday, 3 September 2014

New Uniface, a roadmap

I read an interesting article today, so I thought I'd post a link...

New Uniface, a roadmap

Photo of David Norfolk
Written By: 
Content Copyright © 2014 Bloor. All Rights Reserved.

Link to article: http://www.bloorresearch.com/blog/the-norfolk-punt/2014/9/new-uniface-a-roadmap/

This matches up pretty well with what I was told at a User Event back in January, but with the welcome addition of Uniface 9.7 to the roadmap.

The main themes for Uniface 10 are...

  • Advanced development environment (a brand new IDE, written in Uniface itself!)
  • Agile development methodology
  • Mobile deployment (dedicated team for mobile development)
  • Cloud deployment (this will come later, not in 10.01 or 10.02)

We'll have to wait until Uniface 10.02 for a migration path from earlier versions though, which should be sometime next year.

Summary: Life after Compuware is still looking good for Uniface.

Friday, 4 July 2014

Generating random numbers (again)

A couple of years ago I wrote a post about generating random numbers.  I talked about three different ways of generating a random number (in absence of a $random function in Uniface)...

1) Use a perform to call a DLL (or shared object) that returns a random number.
2) Use $uuid to generate a random number in Uniface.
3) Use a DIY Linear Congruential Generator, built in Uniface.

I found that the perform was the best for performance, and I also stated that it gave a better randomness as well, but I failed to back this up (sorry). 

This issue has raised it's head again, as we're currently trying to get rid of all our own external libraries (DLL/SO) in order to improve maintainability and interoperability.

This time I'm going to test over 1,000,000 iterations (instead of 2,000,000 as before), as there's a little more code inside the loop to store the generated numbers.  Each method will be generating numbers between 0 and 99, and I will then plot the number of times each of these numbers is generated, and report the standard deviation as well - this should give a good idea of the spread.

1) perform: 35.98 seconds (0.036ms per value) - standard deviation: 28.870

2) $uuid42.8 seconds (0.048ms per value) - standard deviation: 28.869

3) DIY: 56.54 seconds (0.057ms per value) - standard deviation: 28.894

As you can see from these charts, and the standard deviation of the numbers generated, all three of these methods have a similarly decent spread.  The time difference hasn't been quite as drastic this time, although that's likely to be due to the extra code in the loop that stores the values for the spread analysis.  Unless you're planning to generate a million random numbers, it's not likely to have a noticeable effect on performance though.

One thing to note though, if you're only concerned with Windows environments, then you don't have a problem, and as long as your DLLs (or shared objects in Unix) are equivalent, then you don't have a problem.  However, the $uuid method does not work as well in Unix environments...

4) $uuid in Unix: (time is not comparable) - standard deviation: 28.833

As you can see, the spread is not uniform at all.  This is because UUID is not sufficiently random in Unix - the first 5 characters seem to be time based, then 3 random characters, then the rest remains constant within the same userver (presumably based on the machine itself and the process ID).  

Summary: As before, perform is the quickest method, but if you're supporting multiple environment configurations then it's likely that building your own random number generator (possibly using $uuid on Windows) is going to make your life much easier.

Additional: There's now a blog post about this on Uniface.info, if you're interested... http://unifaceinfo.com/random-number-generator-uniface/

Wednesday, 18 June 2014

Converting a form to a service and vice versa

I discovered something new and rather interesting yesterday.  And by new, I mean new to me, apparently it's been there "forever"!

I was trying to do some testing for my earlier post on handles, and wanted to run an operation on a mapped server.  However, the operation was on a form, and not a service, which it needs to be to run mapped.  I was just about to create a new service and copy the operation code when I mentioned this off hand in conversation with a colleague of mine, and he mentioned that it was actually very easy to convert a form into a service.

I thought he was about to show me some really clever, well hidden little trick.  Alas, it's right there in the "File" menu, and always has been...

So all you have to do is click that and compile, job done!  You learn something new every day.

Summary: It's really easy to convert a form into a service, or many other things, and vice versa, which you might need to do if you want to run an operation on a mapped server.

Thanks to Mark R for pointing this out for me.

Tuesday, 17 June 2014

What are handles and should I be using them?

The Uniface manuals describes a handle as...
A handle is a reference to an object, such as a component instance, entity, occurrence, or field. Handles can be used to call operations on the referenced object, or to pass these objects as variables and parameters.

If you're used to pointers, you may at first glance think that these are the same, but they're not.  A pointer contains the address of an item in memory and is therefore fixed, whereas a handle is an abstract reference, which means the address can be changed without breaking the reference.  This makes them very useful.

In Uniface you can have "public" or "partner" handles, which allow you to define which types of operations can access them.  They are also datatypes available for local, component and global variables, depending on the scope required.  I'm not going to discuss either of these points in detail, but I think it's handy to know.

As the description in the manual states, you can create a handle that references most Uniface objects, and there are different methods for creating these handles:  Component instances ($instancehandle), entity collections ($colhandle), entity occurrences ($occhandle), fields ($fieldhandle) and also OCX controls ($ocxhandle).  

I'm going to focus on component instances.  More specifically, I'm going to focus on creating new component instances, using newinstance and handles.  

As you might expect, creating a new instance of a form has a little lead time.  Using a handle doesn't stop this lead time, but what it does do, is mean that it only has to happen once.  This means that using a handle for a single form activation is not going to give you any advantage (unless you just love handles!) but when calling operations on the same form multiple times, handles could give you a performance improvement.

This is something like how you could activate a form...

  activate "TEST_FORM".test_oper("Test")

In fact this does an implicit newinstance as well (if it can't already find an instance with the same name as the form) and so explicitly your code could look like this...

  newinstance "TEST_FORM","TEST_FORM"
  activate "TEST_FORM".test_oper("Test")

If you were to replace this with a handle then you may use something like this...

  newinstance "TEST_FORM",theHandle

Notice that in this case we have used a handle instead of an instance name in the newinstance call, and then we can use this handle in place of the activate command.  The -> is referred to by Uniface as the "dereference" operator, and is processed with a precedence of 2 (with only field indirection being higher).

So let's try these two different ways of activating an operation (in this case a very simple operation that simply returns out immediately).  The newinstance command is done before the loop, and the activate (or equivalent handle call) is done inside the loop, 200,000 times...

  • Using activate: 2.56, 2.61, 2.60 (about 2.6 seconds)
  • Using a handle: 2.39, 2.35, 2.39 (about 2.4 seconds)

As you can see, using a handle is marginally quicker, although you'd have to be making a lot of calls before you really noticed the difference.  

I thought that this would be even more useful when the handle references a component instance that is running on another server, such as when a form is mapped to run elsewhere, as the I thought the initial creation of the new instance would take longer here.  So I ran the same operation 20,000 times (after mapping the form)...

  • Using activate: 13.39, 14.35, 12.99 (about 13.5 seconds)
  • Using a handle: 13.56, 13.12, 12.91 (about 13.0 seconds)

Not much difference here either.  Having said that, maybe on a chatty network the difference would be greater.

Please note:  You must return out of the operation that you call, if you exit out then this will destroy the handle and future calls to the operation will fail with a $status of -1.  To clean up a handle you can either set it to 0 or "" (empty string). 

Summary: Handles are abstract references to pretty much any Uniface object.  They can be useful, especially if you want to call the same object multiple times, but they don't seem to have the performance benefits that I was expecting, not when it comes to component instances anyway.

Saturday, 7 June 2014

Sendmessage versus postmessage

If you wanted to send a message from one form to another, you could simply use sendmessage.  This was deprecated in favour of using activate, a command which has become pretty much a catch-all for all sorts of earlier commands, including run, perform, and spawn.  These all still work in Uniface 9.6, but as they were deprecated in Uniface 9.1, it's well worth trying to write these out of your code.

One example of this might be if you have retrieved multiple records in one form, and also have a second form open with related records in it; when you scroll through the occurrences in the first form you might want to post the primary key details through to the second form, so that only the records related to the current occurrence in the first form are displayed.

In order to do this, the first form would need to have code in the "Occurrence Gets Focus" <OGF> trigger, something like this...

  sendmessage "SECOND_FORM","PKEY","FIELD1.ENT=%%FIELD1.ENT%%%"

As you can see, because sendmessage is now deprecated, it is not highlighted as a reserved word.  You also get a compile warning like this...

  warning: 1000 - Deprecated statement 'sendmessage' used

You then need to have some code to receive this message in your second form, which needs to be in the "Asynchronous Interrupt" <ASYS> trigger, something like this...

  the_result = $result ;will be "message"
  message_id = $msgid ;will be "PKEY"
  message_data = $msgdata ;will be "FIELD1.ENT=xxx"

When using sendmessage or postmessage, $result will always be set to "message".  The second value that you define will become the $msgid and then third value will become the $msgdata.  

Please note:  If the <ASYS> trigger in the called component is blank then the application <ASYS> trigger will be called, as defined in the start-up shell.

So what's the difference between sendmessage and postmessage?  Can you simply go through and replace every sendmessage with a postmessage, or should you be using activate instead?

Well the difference is that sendmessage is synchronous and postmessage is asynchronous, which means that potentially there's a delay before the message will be received and processed with a postmessage.  Using an activate would be synchronous but you'd need to rewrite your code, probably moving the code in the <ASYS> trigger into an operation and then calling that from the <OGF> trigger.

Is this potential delay likely to cause a problem though?  Well for starters, can we see it in practice?  The answer to this is "yes", by putting sending code like this somewhere in a form (I went with the <EXEC> trigger)...

  putmess "Before sendmessage"
  putmess "After sendmessage"

  putmess "Before postmessage"
  putmess "After postmessage"

And then putting the receiving code in the <ASYS> trigger, like this...

  putmess "In ASYS: ID=%%$msgid%%%, DATA=%%$msgdata%%%"

We can then run the form and view the message buffer...

  Before sendmessage
  After sendmessage
  Before postmessage
  After postmessage

As you can see, the sendmessage happens between the "before" and "after" (synchronously), whereas the postmessage happens at the very end (asynchronously).  This means that you have to be very careful about changing any sendmessage statements into postmessage statements - the code order will change, and you need to be aware of this and check this will not cause a problem.

As far as my testing has gone, the messages are always received and processed in the order that they were posted, as you may expect from a message queue.  Having said that, I've not found anything in the manuals that states explicitly that this order is enforced, so writing code that relies on the messages being processed in a particular order might not be wise.

Summary: The deprecated sendmessage command was sychronous whereas the postmessage command is asychronous, so you must be careful if you replace one with the other.  However, this is likely to be more straight forward than replacing with the activate command, as the manual suggests.