Saturday, 7 June 2014

Sendmessage versus postmessage

If you wanted to send a message from one form to another, you could simply use sendmessage.  This was deprecated in favour of using activate, a command which has become pretty much a catch-all for all sorts of earlier commands, including run, perform, and spawn.  These all still work in Uniface 9.6, but as they were deprecated in Uniface 9.1, it's well worth trying to write these out of your code.

One example of this might be if you have retrieved multiple records in one form, and also have a second form open with related records in it; when you scroll through the occurrences in the first form you might want to post the primary key details through to the second form, so that only the records related to the current occurrence in the first form are displayed.

In order to do this, the first form would need to have code in the "Occurrence Gets Focus" <OGF> trigger, something like this...


  sendmessage "SECOND_FORM","PKEY","FIELD1.ENT=%%FIELD1.ENT%%%"

As you can see, because sendmessage is now deprecated, it is not highlighted as a reserved word.  You also get a compile warning like this...

  warning: 1000 - Deprecated statement 'sendmessage' used

You then need to have some code to receive this message in your second form, which needs to be in the "Asynchronous Interrupt" <ASYS> trigger, something like this...


  the_result = $result ;will be "message"
  message_id = $msgid ;will be "PKEY"
  message_data = $msgdata ;will be "FIELD1.ENT=xxx"

When using sendmessage or postmessage, $result will always be set to "message".  The second value that you define will become the $msgid and then third value will become the $msgdata.  

Please note:  If the <ASYS> trigger in the called component is blank then the application <ASYS> trigger will be called, as defined in the start-up shell.

So what's the difference between sendmessage and postmessage?  Can you simply go through and replace every sendmessage with a postmessage, or should you be using activate instead?

Well the difference is that sendmessage is synchronous and postmessage is asynchronous, which means that potentially there's a delay before the message will be received and processed with a postmessage.  Using an activate would be synchronous but you'd need to rewrite your code, probably moving the code in the <ASYS> trigger into an operation and then calling that from the <OGF> trigger.

Is this potential delay likely to cause a problem though?  Well for starters, can we see it in practice?  The answer to this is "yes", by putting sending code like this somewhere in a form (I went with the <EXEC> trigger)...


  putmess "Before sendmessage"
  sendmessage "TEST_PERF","MESSAGE","SENDMESSAGE"
  putmess "After sendmessage"

  putmess "Before postmessage"
  postmessage "TEST_PERF","MESSAGE","POSTMESSAGE"
  putmess "After postmessage"

And then putting the receiving code in the <ASYS> trigger, like this...


  putmess "In ASYS: ID=%%$msgid%%%, DATA=%%$msgdata%%%"

We can then run the form and view the message buffer...

  Before sendmessage
  In ASYS: ID=MESSAGE, DATA=SENDMESSAGE
  After sendmessage
  Before postmessage
  After postmessage
  In ASYS: ID=MESSAGE, DATA=POSTMESSAGE

As you can see, the sendmessage happens between the "before" and "after" (synchronously), whereas the postmessage happens at the very end (asynchronously).  This means that you have to be very careful about changing any sendmessage statements into postmessage statements - the code order will change, and you need to be aware of this and check this will not cause a problem.

As far as my testing has gone, the messages are always received and processed in the order that they were posted, as you may expect from a message queue.  Having said that, I've not found anything in the manuals that states explicitly that this order is enforced, so writing code that relies on the messages being processed in a particular order might not be wise.


Summary: The deprecated sendmessage command was sychronous whereas the postmessage command is asychronous, so you must be careful if you replace one with the other.  However, this is likely to be more straight forward than replacing with the activate command, as the manual suggests.

Friday, 6 June 2014

The new Uniface


I read an interesting article today, so I thought I'd post a link...


The new Uniface

David NorfolkBy: David Norfolk, Practice Leader - Development, Bloor Research
Published: 30th May 2014
Copyright Bloor Research © 2014
Logo for Bloor Research
Link to article: http://www.it-director.com/blogs/The_Norfolk_Punt/2014/5/the-new-uniface.html

Also mentioned in this article is the new Uniface branding, which if you follow Adrian Gosbell's twitter account you will see looks like this...




Summary: Life after Compuware is looking good so far for Uniface.



Monday, 2 June 2014

Do comments affect compile time or form size?

A while ago, someone asked on UnifaceInfo.com whether it takes longer to compile a form that has a lot of comments in it than one that doesn't, and Theo Neesksens suggested that maybe I should take a look.  A colleague of mine emailed me the post and I filed it away to respond to when I had a moment.  Well, what can I say, I've been busy!  (And I forgot about it!)

There's an easy way to compile a form programmatically (so that I can wrap timing code around it) and that's using $ude, something like this...

  $result = $ude("compile","form","TEST_FORM")

First I selected an existing form with minimal comments, and compiled it a number of times, noting the size.  Then I added lots of comments and noted those times and sizes as well...


  • Original form (93526 bytes): 41.38, 44.84, 42.03 (average of 42.75 secs)
  • 1000 comment lines (93526 bytes): 44.92, 45.88, 46.61 (average of 45.80 secs)
  • 2000 comment lines (93526 bytes): 46.40, 44.34, 47.76 (average of 46.17 secs)
  • 3000 comment lines (93526 bytes): 47.28, 48.85, 49.31 (average of 48.48 secs)
  • 4000 comment lines (93526 bytes): 49.30, 49.25, 49.19 (average of 49.25 secs)
  • 5000 comment lines (93526 bytes): 49.39, 50.07, 49.96 (average of 49.81 secs)
  • 10000 comment lines (93526 bytes): 54.64, 53.98, 55.16 (average of 54.59 secs)

So as you can see, the size of the compiled form is always the same, so the comments are stripped out, and this process of striping out the comments does take some time.  

I'm not aware of a way to get the individual timings out for each phase, but it seems to me that the comments are removed during the first phase, as this seems to take slightly longer as more comments are added.

On a personal note, I think adding extra comments into your code is a good thing, if it makes it more readable, but better yet, have readable code that doesn't need many comments!  

Summary: Unless you put thousands of lines of comments in, it's not going to have much of an impact on your compile time, but it's certainly not going to have an impact on your compiled file size.

Thanks to Theo Neesksens and Mark R for pointing me in the direction of this one.

Sunday, 1 June 2014

Reducing your memory footprint

One common use for the while loop is to loop through a set of retrieved records.  This can also be done with a for loop, as I discussed in a previous post about types of for loops.  The example I gave was something like this...

  setocc "ent",1
  while $status > 0 )
    ;do something
    setocc "ent",$curocc(ent)+1
  endwhile 

This is nice and simple, and also pretty efficient.  There is one big downfall with it though, and that's memory.  In order to improve performance when you do a retrieve, Uniface works out what the "hitlist" is, but doesn't actually pull all of the retrieved records into memory.  As you loop through, more and more of the hitlist is completed, and the records are pulled into memory.  

To demonstrate this, I retrieved 600 records and then looped through them, using Process Manager to track the memory of the "uniface.exe" program...

  • Initial value: 14.56Mb
  • 600 records retrieved: 14.64Mb (+0.08Mb)
  • Looped to occurrence 100: 14.91Mb (+0.27Mb)
  • Looped to occurrence 200: 15.30Mb (+0.39Mb)
  • Looped to occurrence 300: 15.70Mb (+0.40Mb)
  • Looped to occurrence 400: 16.10Mb (+0.40Mb)
  • Looped to occurrence 500: 16.54Mb (+0.44Mb)
  • Looped to occurrence 600: 16.93Mb (+0.39Mb)

As you can see, retrieving the hitlist is just the first step, and a small one at that, the data is not read in until you loop through the occurrences.

If you had retrieved a rather large amount of data, looping through it like this would continue to hold all of that data in memory.  In the situation where you're processing records one by one, you often no longer need the record after it's been processed, so it's a good idea to discard the record, to free up the memory.  

Each time you discard a record, it removes it from the hitlist and then does an implicit setocc to the next record, which becomes the new current record.  Another thing that discard does is set $status to be the occurrence number, or 0 if there are no records left in the hitlist, just like setocc does.  This allows us to build a very similar loop to the one we had before...

  setocc "ent",1
  while $status > 0 )
    ;do something
    discard "ent",$curocc(ent)

  endwhile

An alternative that I often see is something more like this...

  setocc "ent",1
  while $hits("ent") > 0 )
    ;do something
    discard "ent",1

  endwhile

I'm not a fan of this one though, as $hits can sometimes have performance problems of it's own, caused by it's tendency to complete the hitlist in order to return the count.  

So, let's check how these different methods stack up against each other performance-wise, using the same 600 records as before...

  • Original loop: 6.37, 6.02, 5.90 (around 6 seconds)
  • Discard loop with $curocc: 5.95, 5.95, 5.86 (just under 6 seconds)
  • Discard loop with $hits: 5.95, 6.00, 5.89 (just under 6 seconds)

As you can see, there isn't that much difference in the approaches as far as performance goes.  In this case $hits hasn't caused a problem, but if I remember correctly, it's environment and/or database specific, so that's probably why.  


Summary: Using discard with a large record set is a good idea, because it reduces the memory footprint of the userver, and at no noticeable cost to processing time.  

Monday, 12 May 2014

Handling a JSON Web Token (JWT)

So I've been working on using some Google authentication for a Uniface web application, and it's clever stuff.  However, being the security conscious people that they are, they use a JSON Web Token (JWT) - pronounced "jot", apparently.

To quote the abstract...
The claims in a JWT are encoded as a JSON object that is digitally signed using JSON Web Signature (JWS) and/or encrypted using JSON Web Encryption (JWE).

The Google API documentation is pretty good.  It gives you an endpoint that you can use to verify the token for debugging purposes, but suggests that in production you should be doing the verification locally...
Fortunately, there are well-debugged libraries available in a wide variety of languages to accomplish this.

I did look, of course I did, but did I expect to find Uniface on that list?  No I did not.  They have examples for .NET, Java, PHP, Python and Ruby.  So this is me, trying it the hard way, in Uniface.

I should point out, there are a few caveats to this...

  1. I've not done anything with encrypted tokens (JWEs).
  2. For signed tokens (JWSs) I've not validated the signature - I tried, but I can't get Uniface to do the encryption properly - I have a FrontLine call open about this.
  3. I use an included procedure "json_to_list" in a few places - this is something I'd previously written which uses string manipulation to convert a JSON string into a Uniface list.
  4. This code is provided as is, with no guarantee that it will work, it is merely for demonstration purposes. 

entry jwt_to_list
params
  string pToken : in ;JSON Web Token (JWT)
  string pList : out ;Uniface list of data
endparams
variables
  string vToken,vHeader,vHeaderJson,vHeaderList,vAlgorithm,vTokenMode,vTokenType
  string vEncryption,vKeyId,vKeyUrl,vPartTwo,vPartTwoJson,vPartTwoList,vPartThree
endvariables
 
  ;check parameters
  if ( pToken = "" )
    return -101 ;no token
  endif

  ;split token into 3 parts
  vToken = $replace(pToken,1,".","·;",-1)
  if ( $itemcount(vToken) != 3 )
    return -102 ;token doesn't have 3 parts
  endif
  getitem vHeader,vToken,1
  getitem vPartTwo,vToken,2
  getitem vPartThree,vToken,3
  if ( vHeader = "" | vPartTwo = "" )
    return -103 ;token parts are missing (check third part later, depends on mode)
  endif 

  ;decode header
  vHeaderJson = $replace($replace(vHeader,1,"_","/",-1),1,"-","+",-1)
  vHeaderJson = $encode("USTRING",$decode("BASE64",vHeaderJson))
  if ( $status < 0 | $procerror < 0 | vHeaderJson = "" )
    return -104 ;header could not be decoded
  endif
  call json_to_list(vHeaderJson,vHeaderList)
  if ( vHeaderList = "" )
    return -105 ;header JSON is invalid
  endif

  ;extract header values
  getitem/id vTokenType,vHeaderList,"typ"
  delitem/id vHeaderList,"typ"
  getitem/id vAlgorithm,vHeaderList,"alg"
  delitem/id vHeaderList,"alg"
  getitem/id vEncryption,vHeaderList,"enc"
  delitem/id vHeaderList,"enc"
  getitem/id vKeyId,vHeaderList,"kid"
  delitem/id vHeaderList,"kid"
  getitem/id vKeyUrl,vHeaderList,"jku" 
  delitem/id vHeaderList,"jku"
  if ( vHeaderList != "" )
    return -106 ;unknown header values
  endif

  ;check signature algorithm
  selectcase ( vAlgorithm )
    case "none"
      vAlgorithm = "" ;plaintext token
      vTokenMode = "JWT"
    case "HS256"      
      vAlgorithm = "HMAC_SHA256" ;HMAC using SHA-256 hash
      vTokenMode = "JWS"
    case "HS384"
      vAlgorithm = "HMAC_SHA384" ;HMAC using SHA-384 hash
      vTokenMode = "JWS"
    case "HS512"
      vAlgorithm = "HMAC_SHA512" ;HMAC using SHA-512 hash
      vTokenMode = "JWS"
    case "RS256"
      vAlgorithm = "RSASSA_PKCS1V15_SHA256" ;RSA SSA (PKCS) using SHA-256 hash
      vTokenMode = "JWS"
    case "RS384"
      vAlgorithm = "RSASSA_PKCS1V15_SHA384" ;RSA SSA (PKCS) using SHA-384 hash
      vTokenMode = "JWS"
    case "RS512"
      vAlgorithm = "RSASSA_PKCS1V15_SHA512" ;RSA SSA (PKCS) using SHA-512 hash
      vTokenMode = "JWS"
    case "PS256"
      vAlgorithm = "RSASSA_PSS_SHA256" ;RSA SSA (PSS) using SHA-256 hash
      vTokenMode = "JWS"
    case "PS384"
      vAlgorithm = "RSASSA_PSS_SHA384" ;RSA SSA (PSS) using SHA-384 hash
      vTokenMode = "JWS"
    case "PS512"
      vAlgorithm = "RSASSA_PSS_SHA512" ;RSA SSA (PSS) using SHA-512 hash
      vTokenMode = "JWS"
    case "RAS1_5"
      vAlgorithm = "RSAES_PKCS1V15" ;RSA ES (PKCS)
      vTokenMode = "JWE"
    case "RSA-OAEP-256"
      vAlgorithm = "RSAES_OAEP_SHA256" ;RSA ES (OAEP) using SHA-256 hash
      vTokenMode = "JWE"
    case "ES256","ES384","ES512","RSA-OAEP","A128KW","A192KW","A256KW","dir"
      return -107 ;valid, but not supported by Uniface   
    case "","alg"
      return -108 ;no algorithm (mandatory value)
    elsecase
      return -109 ;unknown algorithm specified
  endselectcase

  ;check mode/third part
  selectcase ( vTokenMode )
    case "JWT"
      if ( vPartThree != "" )
        return -110 ;third part specified in plaintext token
      endif
    case "JWS"
      if ( vPartThree = "" )
        return -111 ;third part missing in signed token
      endif
    case "JWE"
      if ( vPartThree = "" )
        return -111 ;third part missing in encrypted token
      endif
      return -112 ;todo - handle this type
    elsecase
      return -112 ;unknown token mode
  endselectcase

  ;decode second part
  vPartTwoJson = $replace($replace(vPartTwo,1,"_","/",-1),1,"-","+",-1)
  vPartTwoJson = $encode("USTRING",$decode("BASE64",vPartTwoJson))
  if ( $status < 0 | $procerror < 0 | vPartTwoJson = "" )
    return -113 ;second part could not be decoded
  endif
  call json_to_list(vPartTwoJson,vPartTwoList)
  if ( vPartTwoList = "" )
    return -114 ;second part JSON is invalid
  endif

  ;signature mode
  if ( vTokenMode = "JWS" )
    ;todo – validate signature
  endif
 
  ;return data
  if ( vTokenType = "JWS" | vTokenType = "JWE" )
    call jwt_to_list(vPartTwoJson,pList) ;handle nested tokens
  else
    pList = vPartTwoList ;no nesting, just return data
  endif

  return 0
end

Summary:  It is possible to handle JSON Web Tokens (JWTs), but so far I've only looked at plaintext and signed tokens, and I've not managed to validate the signature for signed tokens yet.

Thursday, 8 May 2014

Boolean values in an if statement

Last week I wrote a post about casting in if statements, focusing on the difference between equality and identity.  Well that post started out in my mind as being about boolean values, but then I found I needed to cover that ground first.

Boolean is a rather interesting datatype in Uniface.  The description in the manuals goes like this....

The Boolean data type is interpreted as either TRUE or FALSE. An empty value, and the values 0, F, f, N, and n are interpreted as FALSE. All other values are interpreted as TRUE. 

The Uniface default packing code for boolean stores the value as "T" or "F", but a number of different packing codes can be used...


  • B - Optimum DBMS Boolean default
  • B1 - ASCII Boolean (0 or 1)
  • B2 - ASCII Boolean (F or T) *Uniface default
  • B3 - ASCII Boolean (N or Y)
  • B4 - Binary Boolean (0 or 1)
  • B5 - Binary Boolean (0 or -1)


This means that it is always safest to reference a boolean in an if statement without a relational operator, like this...

  variables
    boolean b
  endvariables

  if ( b )
    putmess "b is True"
  endif
  if ( !b )
    putmess "b is False"

  endif

As empty values are interpreted as false, this example would output "b is False".

This is fine, as long as you know that you've got a boolean datatype.  However, I've seen developers get into trouble when they use this type of if statement (without a relational operator) but with other datatypes.

For example, to check a blank string in javascript, you could easily do something like this...

  var s = "";
  if(!s) {
    alert "s is blank";

  }

If you translate this directly into Uniface, then you get something like this...

  variables
    string s
  endvariables

  s = ""
  if ( !s )
    putmess "s is blank"

  endif

And this works, no problem here.  Empty string is interpreted as false and you get the same result.

Consider this though; what if s is set to "0"?  

In javascript, we would not get the alert - this is because the string is not blank, so quite right!  However, in Uniface we do get the message saying that s is blank, even though it's not.  This is because "0" is interpreted as false.

Summary: In an if statement, if you've got a boolean datatype then don't use a relational operator, but if you've got any other datatype then do use one.