The new Uniface

I read an interesting article today, so I thought I'd post a link...

The new Uniface

By: David Norfolk, Practice Leader - Development, Bloor Research Published: 30th May 2014 Copyright Bloor Research © 2014

Link to article: http://www.it-director.com/blogs/The_Norfolk_Punt/2014/5/the-new-uniface.html

Also mentioned in this article is the new Uniface branding, which if you follow Adrian Gosbell's twitter account you will see looks like this...

Summary: Life after Compuware is looking good so far for Uniface.

Do comments affect compile time or form size?

A while ago, someone asked on UnifaceInfo.com whether it takes longer to compile a form that has a lot of comments in it than one that doesn't, and Theo Neesksens suggested that maybe I should take a look.  A colleague of mine emailed me the post and I filed it away to respond to when I had a moment.  Well, what can I say, I've been busy!  (And I forgot about it!)

There's an easy way to compile a form programmatically (so that I can wrap timing code around it) and that's using $ude, something like this...

  $result = $ude("compile","form","TEST_FORM")

First I selected an existing form with minimal comments, and compiled it a number of times, noting the size.  Then I added lots of comments and noted those times and sizes as well...

• Original form (93526 bytes): 41.38, 44.84, 42.03 (average of 42.75 secs)
• 1000 comment lines (93526 bytes): 44.92, 45.88, 46.61 (average of 45.80 secs)
• 2000 comment lines (93526 bytes): 46.40, 44.34, 47.76 (average of 46.17 secs)
• 3000 comment lines (93526 bytes): 47.28, 48.85, 49.31 (average of 48.48 secs)
• 4000 comment lines (93526 bytes): 49.30, 49.25, 49.19 (average of 49.25 secs)
• 5000 comment lines (93526 bytes): 49.39, 50.07, 49.96 (average of 49.81 secs)
• 10000 comment lines (93526 bytes): 54.64, 53.98, 55.16 (average of 54.59 secs)

So as you can see, the size of the compiled form is always the same, so the comments are stripped out, and this process of striping out the comments does take some time.  

I'm not aware of a way to get the individual timings out for each phase, but it seems to me that the comments are removed during the first phase, as this seems to take slightly longer as more comments are added.

On a personal note, I think adding extra comments into your code is a good thing, if it makes it more readable, but better yet, have readable code that doesn't need many comments!  

Summary: Unless you put thousands of lines of comments in, it's not going to have much of an impact on your compile time, but it's certainly not going to have an impact on your compiled file size.

Thanks to Theo Neesksens and Mark R for pointing me in the direction of this one.

Reducing your memory footprint

One common use for the while loop is to loop through a set of retrieved records.  This can also be done with a for loop, as I discussed in a previous post about types of for loops.  The example I gave was something like this...

  setocc "ent",1

  while ( $status > 0 )

    ;do something

    setocc "ent",$curocc(ent)+1

  endwhile 

This is nice and simple, and also pretty efficient.  There is one big downfall with it though, and that's memory.  In order to improve performance when you do a retrieve, Uniface works out what the "hitlist" is, but doesn't actually pull all of the retrieved records into memory.  As you loop through, more and more of the hitlist is completed, and the records are pulled into memory.  

To demonstrate this, I retrieved 600 records and then looped through them, using Process Manager to track the memory of the "uniface.exe" program...

• Initial value: 14.56Mb
• 600 records retrieved: 14.64Mb (+0.08Mb)
• Looped to occurrence 100: 14.91Mb (+0.27Mb)
• Looped to occurrence 200: 15.30Mb (+0.39Mb)
• Looped to occurrence 300: 15.70Mb (+0.40Mb)
• Looped to occurrence 400: 16.10Mb (+0.40Mb)
• Looped to occurrence 500: 16.54Mb (+0.44Mb)
• Looped to occurrence 600: 16.93Mb (+0.39Mb)

As you can see, retrieving the hitlist is just the first step, and a small one at that, the data is not read in until you loop through the occurrences.

If you had retrieved a rather large amount of data, looping through it like this would continue to hold all of that data in memory.  In the situation where you're processing records one by one, you often no longer need the record after it's been processed, so it's a good idea to discard the record, to free up the memory.  

Each time you discard a record, it removes it from the hitlist and then does an implicit setocc to the next record, which becomes the new current record.  Another thing that discard does is set $status to be the occurrence number, or 0 if there are no records left in the hitlist, just like setocc does.  This allows us to build a very similar loop to the one we had before...

  setocc "ent",1

  while ( $status > 0 )

    ;do something

    discard "ent",$curocc(ent)

  endwhile

An alternative that I often see is something more like this...

  setocc "ent",1

  while ( $hits("ent") > 0 )

    ;do something

    discard "ent",1

  endwhile

I'm not a fan of this one though, as $hits can sometimes have performance problems of it's own, caused by it's tendency to complete the hitlist in order to return the count.  

So, let's check how these different methods stack up against each other performance-wise, using the same 600 records as before...

• Original loop: 6.37, 6.02, 5.90 (around 6 seconds)
• Discard loop with $curocc: 5.95, 5.95, 5.86 (just under 6 seconds)
• Discard loop with $hits: 5.95, 6.00, 5.89 (just under 6 seconds)

As you can see, there isn't that much difference in the approaches as far as performance goes.  In this case $hits hasn't caused a problem, but if I remember correctly, it's environment and/or database specific, so that's probably why.  

Summary: Using discard with a large record set is a good idea, because it reduces the memory footprint of the userver, and at no noticeable cost to processing time.  

Handling a JSON Web Token (JWT)

So I've been working on using some Google authentication for a Uniface web application, and it's clever stuff.  However, being the security conscious people that they are, they use a JSON Web Token (JWT) - pronounced "jot", apparently.

To quote the abstract...

The claims in a JWT are encoded as a JSON object that is digitally signed using JSON Web Signature (JWS) and/or encrypted using JSON Web Encryption (JWE).

The Google API documentation is pretty good.  It gives you an endpoint that you can use to verify the token for debugging purposes, but suggests that in production you should be doing the verification locally...

Fortunately, there are well-debugged libraries available in a wide variety of languages to accomplish this.

I did look, of course I did, but did I expect to find Uniface on that list?  No I did not.  They have examples for .NET, Java, PHP, Python and Ruby.  So this is me, trying it the hard way, in Uniface.

I should point out, there are a few caveats to this...

1. I've not done anything with encrypted tokens (JWEs).
2. For signed tokens (JWSs) I've not validated the signature - I tried, but I can't get Uniface to do the encryption properly - I have a FrontLine call open about this.
3. I use an included procedure "json_to_list" in a few places - this is something I'd previously written which uses string manipulation to convert a JSON string into a Uniface list.
4. This code is provided as is, with no guarantee that it will work, it is merely for demonstration purposes. 

entry jwt_to_list

params

  string pToken : in ;JSON Web Token (JWT)

  string pList : out ;Uniface list of data

endparams

variables

  string vToken,vHeader,vHeaderJson,vHeaderList,vAlgorithm,vTokenMode,vTokenType

  string vEncryption,vKeyId,vKeyUrl,vPartTwo,vPartTwoJson,vPartTwoList,vPartThree

endvariables

 

  ;check parameters

  if ( pToken = "" )

    return -101 ;no token

  endif

  ;split token into 3 parts

  vToken = $replace(pToken,1,".","·;",-1)

  if ( $itemcount(vToken) != 3 )

    return -102 ;token doesn't have 3 parts

  endif

  getitem vHeader,vToken,1

  getitem vPartTwo,vToken,2

  getitem vPartThree,vToken,3

  if ( vHeader = "" | vPartTwo = "" )

    return -103 ;token parts are missing (check third part later, depends on mode)

  endif 

  ;decode header

  vHeaderJson = $replace($replace(vHeader,1,"_","/",-1),1,"-","+",-1)

  vHeaderJson = $encode("USTRING",$decode("BASE64",vHeaderJson))

  if ( $status < 0 | $procerror < 0 | vHeaderJson = "" )

    return -104 ;header could not be decoded

  endif

  call json_to_list(vHeaderJson,vHeaderList)

  if ( vHeaderList = "" )

    return -105 ;header JSON is invalid

  endif

  ;extract header values

  getitem/id vTokenType,vHeaderList,"typ"

  delitem/id vHeaderList,"typ"

  getitem/id vAlgorithm,vHeaderList,"alg"

  delitem/id vHeaderList,"alg"

  getitem/id vEncryption,vHeaderList,"enc"

  delitem/id vHeaderList,"enc"

  getitem/id vKeyId,vHeaderList,"kid"

  delitem/id vHeaderList,"kid"

  getitem/id vKeyUrl,vHeaderList,"jku" 

  delitem/id vHeaderList,"jku"

  if ( vHeaderList != "" )

    return -106 ;unknown header values

  endif

  ;check signature algorithm

  selectcase ( vAlgorithm )

    case "none"

      vAlgorithm = "" ;plaintext token

      vTokenMode = "JWT"

    case "HS256"      

      vAlgorithm = "HMAC_SHA256" ;HMAC using SHA-256 hash

      vTokenMode = "JWS"

    case "HS384"

      vAlgorithm = "HMAC_SHA384" ;HMAC using SHA-384 hash

      vTokenMode = "JWS"

    case "HS512"

      vAlgorithm = "HMAC_SHA512" ;HMAC using SHA-512 hash

      vTokenMode = "JWS"

    case "RS256"

      vAlgorithm = "RSASSA_PKCS1V15_SHA256" ;RSA SSA (PKCS) using SHA-256 hash

      vTokenMode = "JWS"

    case "RS384"

      vAlgorithm = "RSASSA_PKCS1V15_SHA384" ;RSA SSA (PKCS) using SHA-384 hash

      vTokenMode = "JWS"

    case "RS512"

      vAlgorithm = "RSASSA_PKCS1V15_SHA512" ;RSA SSA (PKCS) using SHA-512 hash

      vTokenMode = "JWS"

    case "PS256"

      vAlgorithm = "RSASSA_PSS_SHA256" ;RSA SSA (PSS) using SHA-256 hash

      vTokenMode = "JWS"

    case "PS384"

      vAlgorithm = "RSASSA_PSS_SHA384" ;RSA SSA (PSS) using SHA-384 hash

      vTokenMode = "JWS"

    case "PS512"

      vAlgorithm = "RSASSA_PSS_SHA512" ;RSA SSA (PSS) using SHA-512 hash

      vTokenMode = "JWS"

    case "RAS1_5"

      vAlgorithm = "RSAES_PKCS1V15" ;RSA ES (PKCS)

      vTokenMode = "JWE"

    case "RSA-OAEP-256"

      vAlgorithm = "RSAES_OAEP_SHA256" ;RSA ES (OAEP) using SHA-256 hash

      vTokenMode = "JWE"

    case "ES256","ES384","ES512","RSA-OAEP","A128KW","A192KW","A256KW","dir"

      return -107 ;valid, but not supported by Uniface   

    case "","alg"

      return -108 ;no algorithm (mandatory value)

    elsecase

      return -109 ;unknown algorithm specified

  endselectcase

  ;check mode/third part

  selectcase ( vTokenMode )

    case "JWT"

      if ( vPartThree != "" )

        return -110 ;third part specified in plaintext token

      endif

    case "JWS"

      if ( vPartThree = "" )

        return -111 ;third part missing in signed token

      endif

    case "JWE"

      if ( vPartThree = "" )

        return -111 ;third part missing in encrypted token

      endif

      return -112 ;todo - handle this type

    elsecase

      return -112 ;unknown token mode

  endselectcase

  ;decode second part

  vPartTwoJson = $replace($replace(vPartTwo,1,"_","/",-1),1,"-","+",-1)

  vPartTwoJson = $encode("USTRING",$decode("BASE64",vPartTwoJson))

  if ( $status < 0 | $procerror < 0 | vPartTwoJson = "" )

    return -113 ;second part could not be decoded

  endif

  call json_to_list(vPartTwoJson,vPartTwoList)

  if ( vPartTwoList = "" )

    return -114 ;second part JSON is invalid

  endif

  ;signature mode

  if ( vTokenMode = "JWS" )

    ;todo – validate signature

  endif

 

  ;return data

  if ( vTokenType = "JWS" | vTokenType = "JWE" )

    call jwt_to_list(vPartTwoJson,pList) ;handle nested tokens

  else

    pList = vPartTwoList ;no nesting, just return data

  endif

  return 0

end

Summary:  It is possible to handle JSON Web Tokens (JWTs), but so far I've only looked at plaintext and signed tokens, and I've not managed to validate the signature for signed tokens yet.

Boolean values in an if statement

Last week I wrote a post about casting in if statements, focusing on the difference between equality and identity.  Well that post started out in my mind as being about boolean values, but then I found I needed to cover that ground first.

Boolean is a rather interesting datatype in Uniface.  The description in the manuals goes like this....

The Boolean data type is interpreted as either TRUE or FALSE. An empty value, and the values 0, F, f, N, and n are interpreted as FALSE. All other values are interpreted as TRUE. 

The Uniface default packing code for boolean stores the value as "T" or "F", but a number of different packing codes can be used...

• B - Optimum DBMS Boolean default
• B1 - ASCII Boolean (0 or 1)
• B2 - ASCII Boolean (F or T) *Uniface default
• B3 - ASCII Boolean (N or Y)
• B4 - Binary Boolean (0 or 1)
• B5 - Binary Boolean (0 or -1)

This means that it is always safest to reference a boolean in an if statement without a relational operator, like this...

  variables

    boolean b

  endvariables

  if ( b )

    putmess "b is True"

  endif

  if ( !b )

    putmess "b is False"

  endif

As empty values are interpreted as false, this example would output "b is False".

This is fine, as long as you know that you've got a boolean datatype.  However, I've seen developers get into trouble when they use this type of if statement (without a relational operator) but with other datatypes.

For example, to check a blank string in javascript, you could easily do something like this...

  var s = "";

  if(!s) {

    alert "s is blank";

  }

If you translate this directly into Uniface, then you get something like this...

  variables

    string s

  endvariables

  s = ""

  if ( !s )

    putmess "s is blank"

  endif

And this works, no problem here.  Empty string is interpreted as false and you get the same result.

Consider this though; what if s is set to "0"?  

In javascript, we would not get the alert - this is because the string is not blank, so quite right!  However, in Uniface we do get the message saying that s is blank, even though it's not.  This is because "0" is interpreted as false.

Summary: In an if statement, if you've got a boolean datatype then don't use a relational operator, but if you've got any other datatype then do use one.

Casting in an if statement (equality versus identity)

Uniface is a tightly typed language - this means that when we declare variables, we also declare the datatype, such as string or numeric, like this...

variables

  string s

  numeric n

endvariables

This is very different from a loosely typed language, such as javascript, which does not require variables to even be declared, let alone declare them with a datatype.

As mentioned in previous posts about operators (part one and part two), Uniface uses a single equals (=) as the assignment operator and either single (=) or double (==) equals as the relational operator.  So for equality, you can use either = or == interchangeably.  

Both of these will automatically cast values/variables in order to check their equality.  For example...

s = "1"

n = 1

if ( s = n )

  putmess "True!"

endif

if ( s == n )

  putmess "Also true!"

endif

In both cases, Uniface is saying that the string "1" and the numeric 1 are equal to each other, because it is casting the variables to the same type and then saying that they equal each other.  This is fine, and can be very useful.

Originally I assumed that Uniface would be casting both of the variables to string, but in this case I believe it is actually casting them both to numeric, because this example also works...

s = "001"

n = 1

if ( s = n )

  putmess "True!"

endif

There is a specific section in the manuals about this...

Numeric Data, Empty Strings, and Relational Operators

In general, when two operands are compared using a relational operator and one of those operands is numeric (that is, data type Numeric or Float), the other operand is usually converted into numeric data (for the purpose of the comparison only). There are, however, special considerations when a numeric operand is an empty string ("") or contains a space.

In javascript (for example) there is another operator, a triple equals (===) which can be used to check identity.  In this case, no casting takes place, so the if statement would return false, because the types do not match.  Unfortunately, Uniface does not support this operator, you get a compile error when you try...

Because of this, if you want to check identity in Uniface, you need to be very explicit about making the types the same.  Here are a few examples...

if ( s = $string(n) )

  ;string = string

endif

if ( s = "%%n%%%" )
  ;string = string
endif
if ( $number(s) = n )

  ;numeric = numeric
endif

if ( s*1 = n )
  ;numeric = numeric
endif

Summary: Uniface checks equality but not identity, so you need to be careful about the types of your variables to ensure that the condition behaves the way you expect.