I've started a new weekly APEX newscast

I am really excited to announce that I have started a new weekly newscast called APEX Now.

The pppodcast presents the latest news and events in the world of Oracle Application Express (APEX).

You can catch the very first episode at rebrand.ly/apexnow.

Please feel free to submit any news items or events that you think ought to be highlighted in the newcast. Just send an email to podcast@applinks.co.za.

To access the show notes for Episode 1 of Apex Now, go to www.applinks.co.za/eposide1.

Join me each week for APEX Now.

My APEX Application Got Hacked : Change Passwords Into Passphrases

While I will discuss APEX specifically later in this article, the content here really applies to any web based application.

Most of us as developers fall into a false sense of security when we establish our password policy. I too was guilty of this until I took the time to look into the subject more thoroughly.

Hopefully by the end of this article you will understand the issues with your current password policy and how it can lead to vulnerabilities. Remember that behind your application, there is a database that holds a lot of organisational data as well as client specific information that you are responsible to protect.

All the time in the news we hear of very large corporations that have experienced password breaches resulting in critical customer information being exposed. We are horrified by this and put the blame squarely on the company that we have trusted with our personal information  regardless of  their privacy policy. If big corporations can be hacked, then why can’t our simple user facing application be hacked.

It is a known fact that most users create very bad passwords.

I am not going to insult you by pointing out that the storage of passwords as a string in the database is a monumentally bad idea. None of us I hope would ever do this. In a moment we’ll discuss how to do this in a better way.

Clearly you do need a password policy, but you do need to think carefully about the components of that policy.

In terms of hacking passwords, I will really be discussing offline file hacking as opposed to online directly against your application. Hackers are very intelligent and sophisticated in their methods. The offline file approach would mostly be the method they would use to access your data.

An offline file simply means that your user passwords have been accessed as a file that the hacker can find and use to crack passwords. We'll look at some real examples of how this is done.

Obviously how you store the passwords in the database is a key issue. You absolutely must encrypt the passwords as first line of defense. You need to store the passwords as a hash value.

There are two kinds of hashes – a “one way”  hash and a “two way” hash.

A two way hash means that you have some sort of key or function that allows you to unhash the value stored in the database. Some organisations use this to enable them to send an  email to their user if the user has forgotten their password. This is an unforgivable mistake.  

A one  way hash means that you can never your self  access the value in the database. The hash is created in the  value of the hash can only be accessed by the end user. You do this  by hashing values that only the user knows. One way of doing this is to hash the combination of the username and the password  

So the point is here that you will always want to use a one way hash to ensure that it cannot be unhashed by any database administrator or developer.

Now that we have a good method to store the passwords in the database we can move on to looking at our password policies.

To assess the real strength of the passwords against an attack, I will use the following site:

http://www.howsecureis my password.net

These sorts of sites are not really accurate but it is at least a place to start.

First let’s consider the situation where we have no password policy at all. I’ll use the typical password below:

Password1

When we type this in to the password analysis on the site, this password can be cracked instantly. This is absolutely insecure.

Now what if we require the user to have at least one number and a special character. My password now will be:

p@word1

When we type this in we can see that this password can be cracked in only 2 seconds. Clearly we still have a huge risk.

Well we can changed our policy to require a length of 7  and two numbers and one uppercase letter and one special character. Here we go:

Or@cl3!

Probably looks familiar doesn’t it? A much more complex policy has been used here. When we type this into the site we see that it can be cracked in 7 minutes.

You might be very surprised that the complexity is not so important as is the length of the password. This time my password will only have lower case letters and nothing else.

Here we go:

iliketodrinkwineatlunch

A pretty simple password,right? Type it into the site and look at the result. This password will take 277 trillion years to crack.

So with an 11 or more length requirement really protects us much more and the vulnerability is significantly reduced.

Now it is time to see the really scary stuff.

I have installed a program called Hashcat that is a popular cracking program among hackers.

I will show you how a malicious hacker might actually crack your passwords.

I am going to assume that a list of our hashed passwords has been exposed and the hacker got a hold of this file.This could happen in a lot of ways such as sql injection against your application

The password file I am using contains around 6,500 md5 hashed passwords. You can create your own file by using one of the many online hash generators.

We will try and crack passwords using a number of different attacks.Be aware that I am doing this on my PC with only one GPU. Hashcat uses the GPUss to process. This should make things even a bit more scary.

The first attack we will perform is called a Brute Force attack. It is probably the simplest of all attacks and will deal with passwords that have letters and numbers and are of a length around eight digits.

The attack starts looping through the hashes by first guessing something like “aaaaaa: and then “aaaaaaab: and so on. It will iterate all of the possible combinations and compare them to the hashed password.

We call hashcat and give it the file.

It recovered 1020 hashed passwords at a rate of 139 million hashes attempts  per second.

Here is an example of the passwords it recovered:

The cracked passwords appear after the “:” on each line. You can see that the passwords are not very long nor complex. But this is likely the first attack type a hacker will attempt. The attack completed very quickly and you can see that we matched and cracked a fairly large number of the hashed passwords.

The attack results were pretty good but we still have a ways to go.

Now using the same file we will use a dictionary attack.

A dictionary attack is much more sophisticated than the Brute Force attack

A dictionary may be a list or words in the English language or it may be a list of passwords that have already been cracked through previous breeches. Since hashes always are the same for a specific value, we can compare the hashes to  previously  cracked passwords against our current hash file. These lists of previously cracked passwords come from other very large hacks of large organisation. It is unlikely that a user’s password, or part of it, has been used by another user for another system.

But we don’t want to just use the dictionary. We want to also apply rules to the dictionary values.

For instance, we can toggle between lowercase and uppercase values. Or try using symbols to replace letters. Now the possible combinations greatly increases.

I am using a dictionary of about 12 million words and a rule set of a lot of different rules and combinations. Basically the attack tests each hash by comparing it to the dictionary and then applies the rules . It has a fair bit of work to do so it will take a bit longer.

Let’s run the dictionary attack against our hashed password file : 

This time it recovered another 2,299 hashed at a rate of 20 million hash attempts per second.

Here is an example of the passwords it cracked this time.

You can see that the passwords are a lot longer and complicated.

I could keep going and use other dictionaries and rule sets. In a normal attack the password file may be a lot larger and the passwords more complicated.

Remember the speeds we are getting are based on only one GPU.

Scared yet?

When you use a crack time estimator like the site we have been using here, you should know that they are not entirely accurate. They indeed do give us an indication of how long it might take a hacker to crack our password, but it can only predict in a general way.

The estimators do not really consider the processing speed that may be available to the hacker. Hackers use some powerful tools but they all rely on GPUs or graphic cards. These GPUs are getting more powerful every day. Usually a hacker will have a machine made up of several GPUs rather than just one. The number of crack attempts could be up to several billion per second. The crack time estimators have no way of determining the exact hardware that a hacker might be using.

Another way that the estimators cannot really assess is the sophistication of some of the dictionary attacks is they don’t know the hashing algorithm that the developer has used to store the passwords.

Generally you should understand that md5 is no longer acceptable. There have been so many breeches based on md5 hashes, that it has become very easy to crack them if the dictionary is long enough. It is now best to use SHA256 or SHA512. Oracle 12c has support for using a SHA256 hash.

Here’s what I mean In the earlier dictionary attack we recovered the password “energizer000000”. It was recovered in the first 11 minutes of the attack.

Going back to our estimator site and typing in the password “energizer00000” the result is that it will take 175 thousand years. Not quite as we have seen.

If you consider the password “iliketodrinkredwine” it is made up of the following words:

I like to drink red wine

The phrase will not truly be that difficult to crack. There are several words that appear to be standard phrases:

I like    red wine

In other words, these words would often follow each other in the English language. The associated familiarity of the phrase makes the dictionary attack easier.

But consider the following password:

drawerbufferturtlepalm

It consists of the following individual words:

drawer  buffer turtle palm

These are not very common words and there is no real connection between the words. A dictionary attack is much harder as it would have to combine all four words in exactly this same order.

Now, let’s at this minor change:

draw@er!bufferturtlepalm

A slight change has been made here. Special characters have been added to the phrase but note that they are not used to replace characters but are inserted into the words. The dictionary attack becomes very difficult as the individual words won’t be found exactly as they appear here.

When you replace letters with special characters rather than inserting them into the word, you are not really fooling the hacker. Users, for instance, often replace “a” with @ or an “I” with 1. Hackers know these tricks and they software they use already evaluates in this way.

How then do I deal with things in an APEX Application?

This is exactly the  same for APEX since it accesses key data from our Oracle database and uses public database connections to process data.

When it comes to APEX, the security of passwords really has to do with the authentication. The authentication we use is a way of identifying the user and ensuring the password they use is correct.

There are some preconfigured authentication methods available to us. One of them is to authenticate the user by reference to LDAP. This is a good method since as a user joins or leaves an organisation, the LDAP information is updated. The only issue here is that the user will be able to access all of your applications with the same credentials. You need to control this.

You can create your own custom authentication and this might work best for you as it is flexible and can validate the authentication of the user to address your specific needs including the method by which you encrypt the password for storage in the database.

Now this might be another surprise. As of Apex 18.1, we have another authentication method available to us. We can allow the user to be identified by way of their social media account such as Google or Facebook. In reality, this is a very secure method. In essence you are turning the issue over to them. Each of the social media accounts have really strong password policies and storage. They also, in some instances,  use what is called “two factor verification". This means that they have used two ways to validate the user. Often it may be an OTP or an SMS with a verification code. Since only the user has this information, the validation of the user is much more secure. Generally, most social media accounts store their passwords in quite sophisticated ways and thus may be a good choice for your application.

In addition to your authentication scheme in APEX, you must also define an authorisation scheme. APEX does not require this as default, but you must never overlook the importance of defining an authorisation scheme.

You can think of authorisation in APEX as defining  responsabilities or roles. Think about it. If you do not define specific authorisations, any user that is in LDAP account can access all of your APEX applications and view data or transact.

I usually use a custom authentication as it gives me more control over the password processing as well as an ability to do some fine grain authorisation.

But, here is where my approach is somewhat different and is based on all the information above.

Where I have a “change or reset password page”, I replace the the placeholder of the password field to “Passphrase”.

The password policy I implement is really quite simple :

-- It must be at least 14 characters long
-- It cannot contain the user's first or last name
-- The same passphrase cannot be used twice in a row 
--The passphrase will expire every 30 days.

A lot of web applications are starting to use the concept of a “PassPhrase”. User’s need to be educated on what a passphrase is. So on the page I make sure there is a link that will help the user to understand what it is and how to select one.

The guidance I usually provide is something like this:

-- Your passphrase should be something that you can easily remember and type
-- Choose three or four words 
-- Choose words that are unusual and really don’t belong together
-- You might want to use your favourite band name or something like that for one of the words 
-- You can use symbols as part of the words but insert them rather than replacing letters with them 
-- Here are a few examples but you are not allowed to use these exact ones:

b 
 eyelemonsink@level
tornattackfridgelink

·        - Do not use the same phrase for any other application or web service

Next I maintain a blacklist of words that cannot be used as part of any passphrase. This list may include such words as:

Password

Apex

Login

,,, and so on.

When validating a user’s selected pass phrase, I first check that it does not contain any of the blacklist words, and then make sure that it complies with the policy.

Passphrases, if done correctly are very secure and are less prone to any kind of attack. They are very easy for the user to remember and less likely to be typed with any errors. It will just take a bit of time to educate the user but in my experience they really like it and have provided some very positive feedback on the concept.

Some people recommend that users make use of some sort of password manager in to manage the specific passwords for applications and only require the user to remember one master password. Users can then create very random strings that are difficult to crack. This is a pretty good idea but with APEX I do not use a password manager.

It is my view that with APEX a password manager makes it very difficult, if not impossible, to access a APEX Workspace or an Apex application from any other machine than yours. I often do training and consultation where it is not always possible to use my own machine. If an APEX application is truly a web application, you should be able to access it from anywhere and from any machine.

I hope that at at the very least this article has got you thinking and reassessing how you store passwords in the database and how you set a policy that will be secure now and going forward.

Hackers get more and more advanced in their techniques all the time. We owe it to our users and our customers to protect their access security now and tomorrow. Secure access to your application is a constant process of evaluating your current measures against the current advances in password cracking across the internet.'

Want to see if you have been hacked?

Many applications set your email address as the username for your account. And, sadly, a lot of users tend to use the same password on more than one site.

You can check if your email address has ever been subject to a password crack on any site by going to the following URL:

https://haveibeenpwned.com/

Just type in your email addressed and it will show you if any password associated with the email addressed has ever been compromised.

Thanks Patrick! You lit up the Apex Page Designer for me

As some of you might know, but I suspect not many, I am visually impaired. In some jurisdictions I would be considered legally blind by legislation. I have been visually impaired all of my life. I have been an Apex consultant / developer for years. I have developed Apex application, trained Apex and mentored Apex for over 10 years. I am extremely passionate about Apex.

Very recently Patrick Wolf did a presentation titled "Wow what a Page Designer". You can watch it on Youtube and I have placed the link at the bottom of the post.

In his presentation, Patrick highlights many features of the Page Designer. In specific, he identifies many of the keyboard shortcuts available to move around the Apex workspace and the Page Designer. Some of the things he discusses m ay already been known to you, but for me many of them were new.

The keyboard shortcuts are usually framed as a way to increase your productivity. For me they are an essential part of accessibility. I use a number of specific tools on my laptop to help me in other ways, but moving around the Page Designer with a mouse can sometimes be a bit challenging. The keyboard shortcuts and features like the search spotlight, really enable me to do my job in an easier way. I think that the Oracle Apex development team is doing a great job in ensuring that everyone can equally participate both in developing and using Apex applications.

I  am working currently on an article about how to ensure that your Apex application is fully accessible so that all developers can make this a fundamental part of their design and development process.

Thanks very much Patrick for presenting some of these unique features of the Page Designer. While for some they may be nice to know, for me, they are absolutely critical.

You can view Patrick's presentation  here.

An Interview With Joel R. Kallman

As an Apex developer you won't go very far without hearing the name Joel R. Kallman. Joel is the Oracle's Director of Software Development for Application Express. He is a long time contributor and friend of the Apex community and has participated in many conferences and workshops. Best of all, he is always willing to assist developer as a passionate evangelist of Apex.
I recently had the opportunity to pose a number of questions to Joeal and wanted to share his responses with you. Joel, thank you so much for taking the time for this and for sharing some key information.

Q :         For those who may not know, what is your role as it relates to Oracle APEX?

A:          Good question.  APEX is something that Michael Hichwa (Vice President, Database Tools) and I started back in 1999.  Mike is the inventive and creative one, and I just happened to be employee #1.  He architected the first framework, and I concurrently wrote the first application in this framework, proving and pushing the framework.  Starting from a blank slate, in 3 months time, we developed and deployed a Web Calendar application for all of Oracle (used by 25K people, at the time).  This framework became HTML DB and then ultimately Application Express (APEX), and is used by thousands of customers worldwide.

Today, I am responsible for both the product development and product management of Oracle APEX.  I’m blessed to have a very hard-working, focused, experienced and professional team, and all of us contribute to Oracle APEX.

Q:          What you consider some of the more exciting new features in Apex release 18.1?

A:          A lot of this is presented in our blog post announcement:  https://blogs.oracle.com/apex/announcing-oracle-apex-181.  Probably the two biggest features are Application Features and the enhanced REST support.

For many years, our team has developed and delivered productivity applications as part of Oracle APEX.  Some examples include Group Calendar, Project Tracking, Survey Builder, etc.  In the development of this “fleet” of applications, we found
 ourselves desiring the same features over and and over again – they were common features that someone would want in any APEX application.  So we extended the concept of what you could create from the APEX application builder, and elevated it to higher-level features to be added to your application.  It can consist of multiple pages and components which are employee in an app.  Some good examples include user access control, activity monitoring, user-selectable UI (theme), and user feedback.  Just by checking a box during application creation time, you can include very high-quality, polished application features which we use ourselves.  These features can also be added to existing applications.

APEX 18 is also a dramatic leap forward in the support for remote data sources over REST.  APEX has always had support for REST Web services, but in many cases, the support was programmatic.  In APEX 18, there is now a completely declarative interface to remote data sources over REST.  You can create Web Source Modules (a new shared component type) in APEX 18, and Web Source Modules can be the data source for reports, charts and calendars.  Anyone can use this functionality today on the free site https://apex.oracle.com.  I wrote a recent Oracle Magazine article which includes an interesting (and fun!) step-by-step tutorial.  https://blogs.oracle.com/oraclemagazine/beyond-the-database.

APEX 18 also “unifies” the REST Services with Oracle REST Data Services (ORDS).  There used to be a separate repository for the REST Services that you could create with APEX.  ORDS, which is the next generation of REST Services for the database, had a separate repository.  In APEX 18, these are now unified, and the REST Services that you can create in the REST Workshop of APEX 18 can also be edited via SQL Developer or even in the ORDS-supplied PL/SQL API

Q:          What has been the feedback from the Apex community on release 18.1?

A:          There has been tremendous uptake of APEX 18 by our customers.  It has proven very reliable and stable, and there have been very few upgrade issues from prior versions of APEX.  And in many cases, the applications themselves are faster too.

Q:          Release 18.2 followed quickly after 18.1. Is there anything key to release 18.2 that we ought to know about?

A:          The biggest thing you should take note of is the “cadence” we wish to achieve with APEX releases.  Gone are the days of these very large, monolithic releases of APEX.  Instead, we are opting for a release of APEX every 6 months.  There are a number of more modest features that went into APEX 18.2, but it gets us and our customers in the process of this frequent release process.  To accommodate this, we also made great strides in APEX 18.2 with the near-zero downtime APEX upgrades.

Q:          Some people were running applications in Apex 5.1 and using the compatibility mode to run applications created in older Apex versions. Will these still  work after upgrading?

A:          We still support the same level of compatibility modes as we have done in the past.  However, ideally, people should not rely upon these forever.  The compatibility mode is an opportunity for you to have your applications run in a legacy mode, but there is a very good reason why we introduced a change in behaviour.  With this said, there are many customers who directly upgraded from APEX 4.2 and APEX 5.0 to APEX 18 with literally zero issues.

Q:          I know  that John Snyders has upgraded his IG Cookbook, but can you highlight some of the improvements and new features?

A:          There is new “Copy Don” support, which can be used to copy data from one row to other rows.  There is integrated “Copy to Clipboard” support for a row or cell range selection.  There are new Dynamic Action events which directly support Interactive Grid, namely Mode

Q:        Are JET Charts still the same in this release, or are there new features theras well?

A:          The version of Oracle JET which is included with APEX is updated with every release of APEX.  This allows us to include new data visualizations in APEX, as well as remain on a supported JET release.  In APEX 18, we upgraded to JET 4.2.  And that had the additional effect of upgrading the version of jQuery and jQuery UI included with APEX 18 (jQuery 2.2.3 and jQuery UI 1.10.4).  All native functionality of APEX has been upgraded to work with these newer libraries.
In terms of new charts in APEX 18, we have introduced Gantt Charts, Pyramid and Box Plot.

Q:         Have customers encountered any issues with upgrading that might help us to know?

A:          Nothing really significant.  We always keep the Known Issues of a release up to date, as customers report issues which we believe have broad relevance to the community.  https://www.oracle.com/technetwork/developer-tools/apex/downloads/apex181-known-issues-4478237.html

Q:         Why was there such a dramatic lead in release numbering from Apex 5 to Apex 18?

A:          This was solely to be consistent with the rest of the product naming and numbering with all other products & services from Oracle.  Oracle provides a vast number of service offerings on the Oracle Cloud, and instead of having large, lengthy release cycles, there are releases every quarter, and in many cases, numerous times in a quarter.  Oracle switched to this yearly product numbering for all Oracle Cloud Services.  Other products available both on-premises and on cloud went to this same numbering (e.g., Oracle Database, Oracle REST Data Services, SQL Developer).  We changed the release numbering of APEX to be consistent with this.

Q:         The Oracle Application Express Statement of Direction indicates that there will no longer be patch set releases such as 5.1.4. Does this mean that we will have to wait for the next major release to see issues resolved that developers have identified?

A:          Not particularly.  Between cycles, we will release a cumulative “Patch Set Exception bundle” on My Oracle Support.  We did this for APEX 18.1 and are doing so for APEX 18.2.  We won’t include every bug fix in the patch set bundle, but we will include those which we believe impact a large number of customers.  The upside to this frequent release cycle is you may see new features sooner.  In the past, we could never alter translations or UI in a patch set.  But we can do this in full releases.  So this will ultimately result in us being able to deliver highly sought after functionality sooner.

Q:         So without playing all of your cards, what lies ahead for Apex?

A:          The APEX statement of direction for APEX 19.1 is available at https://apex.oracle.com/sod.  In general, though, things have never looked brighter for the APEX platform or community.

By the way, if you have a moment, you may wish to listen to the recent APEX podcast with Executive Vice President Andy Mendelsohn (this podcast is recorded by the community, and not us).   http://apex.press.


Again, my thanks to Joel.