Undocumented feature - assignment files

I was asked yesterday if I knew about the Uniface function $assignments.  I did and it's something that we started using as soon as it became available (from version 9.something) and it's incredibly useful.  I searched the manuals so that I could send the enquirer the details, but I couldn't find any reference to it.

When a userver starts up, it reads the assignment file in order to get numerous settings.  There are many sections, including a custom section called "logicals", which allows you to add your own assignment files settings, which can be read using the $logical function.  

It is now possible to list all of the logicals at once, like this...

  list = $assignments("LOGICALS")

This returns a Uniface list of all the logicals.  It is possible to return any section of the assignment files in this way, including "paths", "files" and "services_exec".

This was mentioned on the Hacking UnifAce blog over a year ago, but hopefully you find it useful.

Summary: You can access the assignment file settings in code, listed by their section, which can be useful for checking which settings the current userver is using.

Variables as lists

I think you'd be hard pushed to find a Uniface developer that didn't know that a string variable can be used to hold a list of values.  The Uniface list construct uses a gold-semi-colon as the list delimiter, denoted in the manuals as ; but in code looks like ·; - for example...

  s = "27/Sep/2012·;28/Sep/2012·;29/Sep/2012"
      ;27/Sep/2012·;28/Sep/2012·;29/Sep/2012

I spent quite a long time today trying to work out why such a simple thing was not working for me.  It took quite a long time for me to realise that I'd accidentally declared (or rather, re-used) a numeric variable.  This gives a rather different result...

  n = "27/Sep/2012·;28/Sep/2012·;29/Sep/2012"

      ;27·;28·;29

I wasn't previous aware that a numeric variable could hold a list in this way, but it's handy to know! 

The same also works with dates...

  d = "27/Sep/2012·;28/Sep/2012·;29/Sep/2012"

      ;2012092700000000·;2012092800000000·;2012092900000000

Take a look at this screenshot to see the results...

I'm not going to go through all the data types to confirm which ones it works with, but I suspect it would work with all of the simple ones (such as datetime and float) but not with the more complex ones (such as handle and occurrence).

Summary: It's not just string variables that can be used to hold lists; numeric and date variables also can, along with many others I suspect.

Splitting long lines of code

In javascript, for example, white space is pretty much ignored by the interpretor.  You can spread your code over as many lines as you think makes it more readable, just don't forget to indicate the end of the line with a semi-colon.  However, Uniface doesn't allow this, it uses the carriage return as the end of line character.  So what do you do if you have a really long line of code?

Well, for me, I tend to just leave it as a very long line of code.  If it gets too long, I'll ask my boss for an even bigger monitor :) 

I suffer a little with this practice when I'm developing on my laptop though, as I was yesterday.  This reminded me about something that I've seen used before, and that's using a Uniface feature that allows you to split a single line of code over multiple lines, like this...

  if ( value != "1" & value != "2" & value != "3" & %\

       value != "4" & value != "5" & value != "6" & %\

       value != "7" & value != "8" & value != "9" )

    value = "0"

  endif

I'm sure this is something that's documented, but what do you search for to find it?  I certainly couldn't find anything.  So here's a little tip for you, if you prefer to keep those lines a little shorter.  

One situation I have used it in before is with a long SQL command or selectdb statement, as it can help to format it in a similar way to how you might write the SQL in your favourite database tool.  For example...

  selectdb max(field6) from "entity" u_where ( %\

    (field1 = value1) & %\

    (field2 = value2) & %\

    (field3 = value3) & %\

    (field4 = value4) & %\

    (field5 = value5)) to (value6)

EDIT:  Uniface calls this the "Line Continuation Marker" and it is documented as such. 
If anyone knows where this is in the manuals, please leave a comment and let me know.

Summary: It is possible to split a single line of code over multiple lines in Uniface, but you have to do it explicitly.  

Order of operations - part two

It's been pointed out to me that I overlooked a whole page in the Uniface manuals which details the order of precedence for operators - how did I miss that?  Luckily it agrees with my findings, and adds in a few of the Uniface specific functionality (such as indirection)...

Type	Operator	Description	Precedence
Indirection	@	Field Indirection	1
Dereference	->Identifier { }	Struct Dereference and Operation Activation Struct Index	2
Extraction	[ ]	Extraction	3
Indirect dereference	->"SubstitutionString"	Dereference with string substiution	4
Arithmetic Operators	* / %	Multiplication Division Modulus	5
	+ -	Addition Subtraction	6
Relational Operators	< <= != = == >= >	Less than Less than or equal to Not equal to Equal to Equal to Greater than or equal to Greater than	7
Logical Operators	!	Logical NOT	8
	&	Logical AND	9
	|	Logical OR	10

Thanks to Dave R for pointing this out to me.

Summary: Check the manuals before spending hours writing a blog post :)

Order of operations

I was recently asked about the order of operations (or "operator precedence") in Uniface, and I couldn't really answer the question with an certainty, so I thought I'd investigate and prove or disprove my assumptions.  It relates primarily to mathematical expressions, but also to logical conditions, and defines in what order the various components are processed. 

Take for example the following mathematical expression...

x = 10 * 5 + 1

Operator precedence states that multiplication is done before addition, therefore x is 51, not 60.

Wikipedia states that the following order of operations should be true for most programming languages, so my question is, does Uniface comply?

1	()   []   ->   .   ::	Grouping, scope, array/member access
2	!   ~   -   +   *   &   sizeof   type cast ++x   --x	(most) unary operations, sizeof and type casts
3	*   /   %	Multiplication, division, modulo
4	+   -	Addition and subtraction
5	<<   >>	Bitwise shift left and right
6	<   <=   >   >=	Comparisons: less-than, ...
7	==   !=	Comparisons: equal and not equal
8	&	Bitwise AND
9	^	Bitwise exclusive OR
10	|	Bitwise inclusive (normal) OR
11	&&	Logical AND
12	||	Logical OR
13	?:	Conditional expression (ternary operator)
14	=   +=   -=   *=   /=   %=   &=   |=   ^=   <<=   >>=	Assignment operators
15	,	Comma operator

I think it's going to be easiest to prove these in reverse order, so let's take them one by one...

15 - Comma operator

Uniface doesn't have comma operators, you cannot put multiple lines of code on the same line with any separator, as far as I'm aware.

14 - Assignment operators

I wrote a previous post on assignment operators, the arithmetic ones (+= -= *= /= %=) all work in Uniface, but the logical ones (&= |= ^= <<= >>=) do not.  The simplest assignment operator (=) is also used as the comparison operator, there is no differentiation in Uniface.  This means that the comparison operator takes precedence, which be proved with a simple if statement...

x = 0

if ( x = 1 )

  ;if assignment took precedence

endif

askmess "x=%%x%%%" ;shows "x=0"

13 - Conditional expression

This is not valid syntax in Uniface either, you have to do it the long way.  I never really liked this syntax in javascript though anyway, less readable in my opinion.


11/12 - Logical and/or

The logical operators in Uniface are singular (& and |), although the double versions (&& and ||) also work the same way, as I discovered in an earlier post.  I tried to combine these with the assignment operators in order to determine the precedence, but all I got was compile errors.  It seems that assignment operators cannot be used inline.


8/9/10 - Bitwise and/xor/or

Uniface does not have bitwise operators, as previously discussed.  The singular "and" and "or" operations are the logical operators.


6/7 - Comparison

As mentioned above, the equals comparison operation (=) is the same as the simplest assignment operator, and the comparison takes precedence.  It also takes precedence over the logical operators, as this if statement proves...

x = 0

if ( x = 1 | 1)

  x = 1 ;if ((x = 1) | 1)

else

  x = 0 ;if (x = (1 | 1))

endif

askmess "x=%%x%%%" ;shows "x=1"

5 - Bitwise shift

Uniface still doesn't have bitwise operators :)


4 - Addition/subtraction

It's simple enough to show that the addition (+) and subtraction (-) operators take precedence over the comparison operator (=), using another if statement...

x = 0

if ( x = 0 + 1 )

  x = 1 ;if (x = (0 + 1))

else

  x = 0 ;if ((x = 0) + 1)

endif

askmess "x=%%x%%%" ;shows "x=0"

3 - Multiplication/division/modulo

It's also simple to show that the multiplication (*), division (/) and modulo (%) operators take precedence over the addition and subtraction operators...

x = 6 * 1 - 1

;(6 * 1) - 1 = 5

;6 * (1 - 1) = 0

askmess "x=%%x%%%" ;shows "x=5"

2 - Unary/type cast

Uniface doesn't support all the unary operators, in fact I'm not sure what all of them do.  The not (!) operand is easy enough to test though...

x = ! 0 + 1

;(! 0) + 1 = 2

;! (0 + 1) = 0

askmess "x=%%x%%%" ;shows "x=0" but would have expected "x=2"

Surprisingly this one comes up with the opposite result than was expecting, indicating that addition (+) actually takes precedence over not (!).  We know that Uniface is good at type casting though, and will always try to cast a string as a numeric when doing arithmetic operations...

x = "1" * 2

askmess "x=%%x%%%" ;shows "x=2"

We can also show the negative unary operator (-) takes precedence over the subtraction operator (-), despite these being the same, like this...

x = -1 - 2

;(-1) - 2 = -3

;-(1 - 2) = 1

askmess "x=%%x%%%" ;shows "x=-3"

1 - Grouping

Uniface doesn't have arrays or members, and scope is handled explicitly with end statements.  It does however use brackets for grouping, as we've seen through these examples, they can be used to change the order of precedence, as they are always done first.


Summary: It seems that Uniface is a relatively simple in the sense that it doesn't do complicated inline nesting, nor any bitwise operations.  With the exception of the not operator (!), it seems to follow the standard order of operations though.  Personally I prefer to use lots of brackets in order to make the order of precedence more obvious and easier to read, without having to remember these rules.

Scanning is slow

I'm always complaining when I see people using a scan or $scan when they don't need to.  Yes, it can be very useful, sometimes it's unavoidable, but here's an example of when it should not be used.

If I asked you to count how many times a substring appeared within a string, you might think about doing it this way...

  temp = list

  total = 0

  scan temp,"ABC"

  while ( $result > 0 )

    total = total+1

    temp = temp[$result+3]

    scan temp,"ABC"

  endwhile

It's perfectly logical code, it looks through the string, scanning for the substring, counting each iteration.  I've written this code myself, a few years back, and didn't think anything of it.  I recently encountered this code that I'd written, it happens to be part of an import process I use quite often.  However, on this particular day, I was importing 10,000 records - far more than usual.  Whilst I was waiting over an hour for this to import, I decided to check the code.

I noticed that I was using a scan and thought for a moment about what alternatives there were.  The first one I thought of seemed a little strange, but I was sure it would work, so I gave it a go.  This is what it was...

  temp = $replace($replace(list,1,"·;","",-1),1,"ABC","·;",-1)

  total = $itemcount(temp)-1

As you can see, I'm first removing an list delimiters (gold-semi-colon characters) from the string, and then replacing the substring with the list delimiter instead.  This now means that I have a Uniface list, and I want to know how many of these delimiters there are in the string.  This easiest way to do this is use $itemcount to count the number of items, and then deduct one, as there's always one more item than there are delimiters.  This worked a lot quicker!

I've reproduced this for testing, using a string with 500 occurrences of the substring, and performing the count 500 times...

• scan = 00:45.01, 00:43.70, 00:44.31 (just under 45 seconds)
• list = 00:00.86, 00:00.84, 00:00.84 (under 1 second)

As you can see, quite a staggering difference.  I hope you'll think twice before using scan again!  Obviously the loop and the rebuilding of the string is contributing, but I hope this is still a convincing argument.

Summary: Scanning a string can be essential, but it's a very costly function, so it's well worth thinking about an alternative approach.

Generating random numbers

One thing that is always difficult in a system is generating a truly random number.  Computers aren't random, they're very logical, therefore this is inherently difficult.  Having said that, there's always a way to calculate a number which is "random enough".  There is no function for this in Uniface, so I'm going to look at a few different ways to achieve this.


1) perform - the only way we could find to do this originally was to build a random number function in C++ and then call out to it from our Uniface program.  Something like this...


  perform "GetRandomNumber" ;call 3gl function which returns 0-32767
  rand = $1 / 32767


2) $uuid - since the Uniface Unique Identifer function was added, this has given an alternative method.  This is largely based on the current timestamp and either includes the processor ID or the ethernet address, depending on the operating system you are using.  The value returned is a 32 character hexadecimal string, so we need to remove the non-numeric characters.  

  rand = "0.%%$replace($replace($uuid,1,'&',"",-1),1,"-","",-1)%%%" * 1

We actually found that this was not random enough on non-Windows systems, as the last part of the identifier is the same throughout each transaction, so we have used characters from 3 identifiers.


3) DIY - you could also create your own random number generator, after all, these are just mathematical formulas.  The C++ "rand" function that we utilise in method (1) is a simple Linear Congruential Generator.  This takes an initial seed value and then uses it to create the next number in the sequence, which is then used as the seed for the next number.  The key is finding a combination of values that gives an evenly distributed spread of numbers, to ensure that the numbers appear suitably random.


  $$rand = ((214013 * $$rand) + 2531011) % 4294967296
  rand = $$rand / 4294967296


As you can see, this relies on the seed value already being populated, which I've stored in a global register in this example.  This could be set in the application shell execute trigger, maybe using $uuid or a time based numeric.  


Often these algorithms return a subset of the bits in order to improve the spread, but it is not possible to do extraction at the bit level in Uniface, as far as I'm aware.  Another algorithm that is popular (and generally considered better) is a Mersenne Twister, but this uses bit-shifting techniques that I don't think are possible in Uniface either.


So let's test the performance of these different methods of 2,000,000 iterations...


1) perform = 00:10.00, 00:10.00, 00:10.01 (10 seconds)
2) $uuid = 00:32.31, 00:32.44, 00:32.39 (over 32 seconds)
3) DIY = 00:34.75, 00:34.72, 00:34.72 (under 35 seconds)


As you can see, the original perform is the quickest method (although we've found that generally using a 3GL function does not hold up very well under load and these tests are only as a single user).  It can be hard to support a 3GL function across multiple platforms, but this solution is mathematically the most random method.  Out of the alternatives, $$uuid is quite simple but does not give a good spread of random numbers, not compared with the DIY method.  


It should be emphasised that none of these methods are truly random, and therefore should not be used for cryptographic purposes.  They should be suitable for simple things though, like simulating a dice throw.


Hopefully one day Uniface will provide it's own $rand or $random function - a native function should perform the best and would hopefully be implemented in a way that was suitably random with a decent spread.


Summary: If it's feasible to use a perform then this is the best way to go, both for speed and randomness.  However, you may wish to consider building your own random number generator, possibly using a Linear Congruential algorithm.

User-defined functions

One of the things which I initially found a little odd, although very simple to grasp, was the way that Uniface allows you to create modules of code as an entry, not a function.  This entry has any number of parameters (well, there probably is a limit) which can be defined as "in", "out", or "inout".  The only thing that is returned from this entry is a numeric status, which you return and this sets $status accordingly.  I think it's fairly common that developers use a status of "0" to indicate successful and positive for additional successful status, then a negative status for errors.  Any other type of value could be returned in an "out" or "inout" parameter.


This is a very simple mechanism and it works nicely.  However, sometimes it can lead to some rather verbose code, as you have to put the call to the entry on a separate line, you can't nest the call within an if or while statement as part of the condition, for example.  Here's a simple entry which converts kilometres into miles...


  entry kms_to_mls_entr
  params
    numeric kms : in
    numeric mls : out
  endparams
    mls = kms*5/8
    return 0
  end

To convert two values using the entry and then add them, it takes 3 lines, like this...

  call kms_to_mls_entr(kms1,mls1)

  call kms_to_mls_entr(kms2,mls2)

  total = mls1 + mls2

From version 9 onwards (sorry, I can't remember the minor version number) you have another option.  It took me a while to find any information in the manuals, but it's there; they're called "User-Defined Functions".  Here's an example...

  entry kms_to_mls_func

  returns numeric

  params

    numeric kms : in

  endparams

    return kms*5/8

  end

Notice that there is a returns statement before the params, which determines what datatype should be returned.  In this case it is numeric, but it doesn't need to be.  In this case, to convert the two values using this function, it takes a single line...

  total = kms_to_mls_func(kms1) + kms_to_mls_func(kms2)

This can make the code a bit neater.  But of course I'm obsessed with performance, so my next step is the test these two methods over 2,000,000 iterations...

• entry = 00:17.74, 00:17.69, 00:17.60 (under 18 seconds)
• function = 00:13.64, 00:13.64, 0:13.67 (under 14 seconds)

So as you can see, the code is more concise and it performs better.  It also relies on less variables defined, so it's a win-win all round really.  You can even do this with a global procedure.

A bit of a side note; I thought you always had to specify a datatype when defining a parameter, and that this was always a local variable.  I discovered recently that you can also use a component variable or a painted field name, in which case you don't specify the datatype.  For example...


  entry kms_to_mls_entr
  params
    numeric kms : in
    $miles$ : out
  endparams
    $miles$ = kms*5/8
    return 0
  end

Summary: It is possible to create a "user-defined function", which is an entry that returns a specific datatype, allowing you to create more concise code, that also performs better. 

Types of for loops - part two

In part one I talked about the forlist statements and how these could be used to iterate through Uniface lists for easily.  The next thing I want to talk about is looping through entity occurrences. Personally I've always done this using a combination of setocc and $curocc, something like this...

  setocc "ent",1

  while ( $status > 0 )

    ;do something

    setocc "ent",$curocc(ent)+1

  endwhile

This has the advantage of not needing to use any variables for the loop.  However, it may be better for performance if a similar loop was used, but using variables to control it...

  count = 0

  stat = $hits(ent)

  while ( count < stat )

    count = count+1

    setocc "ent",count

    ;do something

  endwhile

Another alternative would be to use the new statement forentity, which was also added in Uniface 9.5...

  forentity "ent"

    ;do something

  endfor

As you can see, the code is much more concise.  There is no need to initialise the count variable or  $status, everything is done as part of the forlist statement, and the incrementing and extracting are done automatically.  The "count" variable is optional, if you don't need it then you don't need to include it.

So let's test these three blocks of code over 65,000 iterations...

• while ($curocc) = 00:00.47, 00:00.47, 00:00.47 (about half a second)
• while (variables) = 00:04.62, 00:04.25, 00:05.10 (about 5 seconds)
• forentity = 00:00.37, 00:00.34, 00:00.35 (about a third of a second)

As you can see, the new forentity is more concise code and also performs better, fairly significantly over some alternatives.

I was surprised by how slow the second method was compared to the first.  The only explanation I have for this is that $hits is taking a long time to complete the hit-list before the loop starts, rather than completing the hit-list as it goes through the loop.  Turns out I’ve been doing it a pretty efficient way all along, but I like the simplicity of the new forentity statement.  Which leads me to an almost identical summary as in part one. 

Summary: Whilst I have previously always used while loops, I shall now be considering switching the forentity loops, for iterating through entity occurrences.