Showing posts with label google. Show all posts
Showing posts with label google. Show all posts

Thursday, 23 October 2014

Fetching web content

I've been working on fetching web content recently, mostly for building authentication workflows, using the likes of Facebook and Google to act as authentication services.  I'm going to go into detail about how I've done those, not in this post at least, but more simply how I made the requests from my Uniface service form, out to those external services. 

There are a number of different HTTP methods.  Generally speaking, when you are retrieving data only then the method is "GET", whereas if you are sending data that may perform an action as well then the method is "POST".  There are others, but I'm going to stick with these two for now, as they are more commonly used.

In fact, I'm going to start with "GET" first.  

The first thing you need to do is consider the character set.  Your system may be set in different character sets, but you need to make sure the character set of your request matches the service you are calling, in my case it was UTF-8...

  vCharSet = $sys_charset ;backup current setting for later
  $sys_charset "UTF8"   ;set to the character set we need

We then want to create a new instance of the "UHTTP" component - this is the Uniface component that is going to do most of the hard work for us...

  newinstance "UHTTP",vHandle
  if $status < 0 & $procerror < 0 )
    $sys_charset = vCharSet ;restore character set
    return -101             ;error handling!
  endif

Once you've got your instance, the next step is to define how the component responds to mismatched or expired certificates, and how it calculates the content length.  By default it will error if there is a certificate error and you have to manually calculate the content length yourself.  It's a binary switch, so to switch them all off we do the following...

  activate vHandle.SET_FLAGS(7)
  if $status < 0 & $procerror < 0 )
    $sys_charset = vCharSet ;restore character set
    return -102             ;error handling!
  endif

We're now ready to make the request.  In this example, I'm just going to grab the Google homepage...

  activate vHandle.SEND("http://www.google.co.uk","GET","","","",vContent,vResponse)
  if $status < 0 & $procerror < 0 )
    $sys_charset = vCharSet ;restore character set
    return -103             ;error handling!
  endif

The parameters are:

  1. URL (string : in) - the URL you're sending the request to.
  2. Method (string : in) - the method being used (this is not checked by "UHTTP").
  3. Username (string : in) - if you're using a secure URL, you may need to populate this.
  4. Password (string : in) - again, you may need to populate this.
  5. Headers (string : inout) - the HTTP request headers in and the response headers out.
  6. Content (string : inout) - in the case of a "GET" this is out only and the page contents.
  7. Response (string : out) - the HTTP response headers.
The $status should be set to 200 for a successful response (to match the HTTP status code for success) or it may be set to 1.  If it is set to 1 then this means that the content was larger than the parameter limit (10Mb) and therefore you will need to get the rest of the content from the buffer, like this...


  while $status = 1 )
    activate vHandle.READ_CONTENT(vExtra)
    if $status < 0 & $procerror < 0 )
      $sys_charset = vCharSet ;restore character set
      return -104             ;error handling!
    endif
    vContent = "%%vContent%%vExtra%%%"
  endwhile


You now have the full page contents, but don't forget to restore the character set...

  $sys_charset = vCharSet ;restore character set

Doing a "POST" is very similar, except for a couple of differences:
  1. To replicate a web browser "POST", which you are usually doing, you need to make sure the 5th parameter of the SEND call is populated with the following header.. "Content-Type=application/x-www-form-urlencoded"
  2. The 6th parameter of the SEND call needs to be populated with the data that you are sending in your request.  If this is larger than 10Mb then you will need to have a WRITE_CONTENT loop (similar to the READ_CONTENT loop) before the SEND, in order to populate the buffer with the full contents.
This all works rather well.  However, I've found two major limitations along the way.

Firstly, it was not possible to send a request to a URL which had a colon (:) character after the protocol.  For example, when trying to get a user's LinkedIn ID you would use the following URL...

  "https://api.linkedin.com/v1/people/~:(id)?oauth2_access_token=%%vToken%%%"

The colon (:) character after "/people/~" broke the URL.  Luckily I'm using the past tense here, this was fixed in patch X504 (for Uniface 9.6.05).  This works fine now we've installed the patch.

Secondly, the observant of you may have noticed the 6th parameter of the SEND call is defined as a string.  This is great for grabbing HTML page source from a website, or even doing most webservice calls, as these tend to return either XML and JSON strings of data.  However, it makes creating something like a Dropbox interface impossible, or at least very limited.  You can grab text files, but nothing else!  No images, no Office documents, no PDFs.

Unfortunately there's no word from Uniface on when this second issue might be resolved.

Summary: It's possible to use the "UHTTP" component to request text data, either from a web page or a webservice.  Just don't expect it to work with binary files, yet!

Monday, 12 May 2014

Handling a JSON Web Token (JWT)

So I've been working on using some Google authentication for a Uniface web application, and it's clever stuff.  However, being the security conscious people that they are, they use a JSON Web Token (JWT) - pronounced "jot", apparently.

To quote the abstract...
The claims in a JWT are encoded as a JSON object that is digitally signed using JSON Web Signature (JWS) and/or encrypted using JSON Web Encryption (JWE).

The Google API documentation is pretty good.  It gives you an endpoint that you can use to verify the token for debugging purposes, but suggests that in production you should be doing the verification locally...
Fortunately, there are well-debugged libraries available in a wide variety of languages to accomplish this.

I did look, of course I did, but did I expect to find Uniface on that list?  No I did not.  They have examples for .NET, Java, PHP, Python and Ruby.  So this is me, trying it the hard way, in Uniface.

I should point out, there are a few caveats to this...

  1. I've not done anything with encrypted tokens (JWEs).
  2. For signed tokens (JWSs) I've not validated the signature - I tried, but I can't get Uniface to do the encryption properly - I have a FrontLine call open about this.
  3. I use an included procedure "json_to_list" in a few places - this is something I'd previously written which uses string manipulation to convert a JSON string into a Uniface list.
  4. This code is provided as is, with no guarantee that it will work, it is merely for demonstration purposes. 

entry jwt_to_list
params
  string pToken : in ;JSON Web Token (JWT)
  string pList : out ;Uniface list of data
endparams
variables
  string vToken,vHeader,vHeaderJson,vHeaderList,vAlgorithm,vTokenMode,vTokenType
  string vEncryption,vKeyId,vKeyUrl,vPartTwo,vPartTwoJson,vPartTwoList,vPartThree
endvariables
 
  ;check parameters
  if ( pToken = "" )
    return -101 ;no token
  endif

  ;split token into 3 parts
  vToken = $replace(pToken,1,".","·;",-1)
  if ( $itemcount(vToken) != 3 )
    return -102 ;token doesn't have 3 parts
  endif
  getitem vHeader,vToken,1
  getitem vPartTwo,vToken,2
  getitem vPartThree,vToken,3
  if ( vHeader = "" | vPartTwo = "" )
    return -103 ;token parts are missing (check third part later, depends on mode)
  endif 

  ;decode header
  vHeaderJson = $replace($replace(vHeader,1,"_","/",-1),1,"-","+",-1)
  vHeaderJson = $encode("USTRING",$decode("BASE64",vHeaderJson))
  if ( $status < 0 | $procerror < 0 | vHeaderJson = "" )
    return -104 ;header could not be decoded
  endif
  call json_to_list(vHeaderJson,vHeaderList)
  if ( vHeaderList = "" )
    return -105 ;header JSON is invalid
  endif

  ;extract header values
  getitem/id vTokenType,vHeaderList,"typ"
  delitem/id vHeaderList,"typ"
  getitem/id vAlgorithm,vHeaderList,"alg"
  delitem/id vHeaderList,"alg"
  getitem/id vEncryption,vHeaderList,"enc"
  delitem/id vHeaderList,"enc"
  getitem/id vKeyId,vHeaderList,"kid"
  delitem/id vHeaderList,"kid"
  getitem/id vKeyUrl,vHeaderList,"jku" 
  delitem/id vHeaderList,"jku"
  if ( vHeaderList != "" )
    return -106 ;unknown header values
  endif

  ;check signature algorithm
  selectcase ( vAlgorithm )
    case "none"
      vAlgorithm = "" ;plaintext token
      vTokenMode = "JWT"
    case "HS256"      
      vAlgorithm = "HMAC_SHA256" ;HMAC using SHA-256 hash
      vTokenMode = "JWS"
    case "HS384"
      vAlgorithm = "HMAC_SHA384" ;HMAC using SHA-384 hash
      vTokenMode = "JWS"
    case "HS512"
      vAlgorithm = "HMAC_SHA512" ;HMAC using SHA-512 hash
      vTokenMode = "JWS"
    case "RS256"
      vAlgorithm = "RSASSA_PKCS1V15_SHA256" ;RSA SSA (PKCS) using SHA-256 hash
      vTokenMode = "JWS"
    case "RS384"
      vAlgorithm = "RSASSA_PKCS1V15_SHA384" ;RSA SSA (PKCS) using SHA-384 hash
      vTokenMode = "JWS"
    case "RS512"
      vAlgorithm = "RSASSA_PKCS1V15_SHA512" ;RSA SSA (PKCS) using SHA-512 hash
      vTokenMode = "JWS"
    case "PS256"
      vAlgorithm = "RSASSA_PSS_SHA256" ;RSA SSA (PSS) using SHA-256 hash
      vTokenMode = "JWS"
    case "PS384"
      vAlgorithm = "RSASSA_PSS_SHA384" ;RSA SSA (PSS) using SHA-384 hash
      vTokenMode = "JWS"
    case "PS512"
      vAlgorithm = "RSASSA_PSS_SHA512" ;RSA SSA (PSS) using SHA-512 hash
      vTokenMode = "JWS"
    case "RAS1_5"
      vAlgorithm = "RSAES_PKCS1V15" ;RSA ES (PKCS)
      vTokenMode = "JWE"
    case "RSA-OAEP-256"
      vAlgorithm = "RSAES_OAEP_SHA256" ;RSA ES (OAEP) using SHA-256 hash
      vTokenMode = "JWE"
    case "ES256","ES384","ES512","RSA-OAEP","A128KW","A192KW","A256KW","dir"
      return -107 ;valid, but not supported by Uniface   
    case "","alg"
      return -108 ;no algorithm (mandatory value)
    elsecase
      return -109 ;unknown algorithm specified
  endselectcase

  ;check mode/third part
  selectcase ( vTokenMode )
    case "JWT"
      if ( vPartThree != "" )
        return -110 ;third part specified in plaintext token
      endif
    case "JWS"
      if ( vPartThree = "" )
        return -111 ;third part missing in signed token
      endif
    case "JWE"
      if ( vPartThree = "" )
        return -111 ;third part missing in encrypted token
      endif
      return -112 ;todo - handle this type
    elsecase
      return -112 ;unknown token mode
  endselectcase

  ;decode second part
  vPartTwoJson = $replace($replace(vPartTwo,1,"_","/",-1),1,"-","+",-1)
  vPartTwoJson = $encode("USTRING",$decode("BASE64",vPartTwoJson))
  if ( $status < 0 | $procerror < 0 | vPartTwoJson = "" )
    return -113 ;second part could not be decoded
  endif
  call json_to_list(vPartTwoJson,vPartTwoList)
  if ( vPartTwoList = "" )
    return -114 ;second part JSON is invalid
  endif

  ;signature mode
  if ( vTokenMode = "JWS" )
    ;todo – validate signature
  endif
 
  ;return data
  if ( vTokenType = "JWS" | vTokenType = "JWE" )
    call jwt_to_list(vPartTwoJson,pList) ;handle nested tokens
  else
    pList = vPartTwoList ;no nesting, just return data
  endif

  return 0
end

Summary:  It is possible to handle JSON Web Tokens (JWTs), but so far I've only looked at plaintext and signed tokens, and I've not managed to validate the signature for signed tokens yet.

Tuesday, 15 May 2012

Hello and welcome

I have been a Uniface developer for the past 7 years, starting with Uniface 8.x and now currently working with Uniface 9.4.  


One thing that can be difficult about working with Uniface is that Google is often not very helpful, you get stuck and there really isn't anywhere to turn.  I have decided it's about time this changed, and this blog is my own small way of doing this.  


Over the next few months I plan to think back over my 7 years so far, and try to write a few articles detailing the tricks and tips that I have come up with, or stolen from colleagues.


I hope someone out there, someday, finds this blog useful.  You never know, it might be me!