+4

Micromine 2016 Feature Highlight -- Dynamic Field Expressions

fbilki (Moderator / Admin (AUS)) 7 years ago in General updated by Nikolay Karakashov 7 years ago 11

Contents

Introduction

Logical operations

Labelling: numbers and text

Expression Editor

Filters

Calculations

Next/previous record

Colour construction

Highlight/use every nth record

Conclusion



Introduction

Have you ever been frustrated when you tried to:

  • Compare two file fields
  • Display every 10th record from a large file
  • Combine separate red, green, and blue values into a colour that Micromine can display
  • Perform a calculation that needs the value from the previous or next record
  • Construct a complicated label from multiple fields?

Although these tasks are all relatively simple they require temporary fields to get the job done, forcing you to modify the file. This can be a problem if you don’t have the necessary permissions or if the file needs to stay in a standard format. In Micromine 2016 you can do the job much more simply and on-the-fly by creating a dynamic field expression.


Field expressions are like equations that define a relationship between one or more file fields, constants, functions and operators. You can use expressions anywhere you’d supply an input fieldname, which means they work in filters, colour sets, labels and calculations, all without altering the input file.


Here’s an example:


=[K2O]-[NA2O]


This simple expression calculates the difference between fields called K2O and NA2O. The equals sign (=) tells Micromine that this is an expression instead of a regular file field, and the square brackets indicate each fieldname.


The square brackets aren’t compulsory but they do make it easier to identify fieldnames in a complex expression. However, they become compulsory if your fieldnames include spaces or reserved characters like ‘+’ or ‘/’.


You can use this expression in pretty much any input field response, such as a Colour field. The best part when you assign values to a colour set based on this expression is that you get numbers that span the full range of differences between the fields, not their original values.


Image 1037




Logical operations

What if you simply want to show if K2O is greater than NA2O? Easy:


=[K2O] > [NA2O]


This is an example of a logical comparison that returns true (1) when K2O is greater than NA2O, and false (0) when it is not. A corresponding (assigned) colour set runs from zero to one (I manually relabelled the two colours in the screenshot).


Image 1038




So far I’ve only looked at numbers, but what about if you want to label the result? Try this:


=if([K2O] > [NA2O]) then
    "Potassic"
else
    "Sodic"
endif


This is a classical if-then-else statement. If K2O is greater than NA2O the text will be “Potassic”, otherwise it will be “Sodic”. You must always include the endif statement. (You can also use fi – as in finished.)


Image 1039




Labelling: numbers and text

Using the preceding expression as a Text Field in a Vizex Point layer will display the labels “Potassic” or “Sodic” at each point regardless of the original numerical values.


You can optionally use the pound/hash symbol (#) to tell Micromine to convert numerical values to text. Although the # isn’t compulsory, it’s a good idea to include it whenever you want to guarantee text output. This is especially true when you’re combining numbers and text, because the ‘+’ operator concatenates text strings whenever you “add” them. For example:


=3 + 4

produces the number 7, whereas

=#(3 + 4)

produces the text “7”, and

=#3 + 4

produces the text “34”, which is simply the concatenation of 3 and 4.


This powerful feature makes it easy to create complex labels by combining any number of separate file fields, but it’s easy to become confused if some of those fields contain numbers. For example, when looking at a printed map, how would you know if the label “7” or the label “34” was correct?


A simple rule of thumb for using the # is to keep its scope as small as possible, by only applying it to numbers or numerical expressions you want to format as text.


You can really go to town with expression-based labelling, from really basic labels like:


="Area = " + #[AREA] + " sq m"

(note the space after the = and before the sq m)


To this:


=if([NA2O] > 0) then
    "K:Na ratio=" + " " + #([K2O] / [NA2O])
else
    ""
endif


This expression combines a bunch of different elements to display the K/Na ratio of the input data. To avoid division by zero the label is conditionally applied where [NA2O] > 0. For non-zero values the label is then constructed from three separate parts: the "K:Na ratio=" prefix, concatenated with an extra space + " " , concatenated with the calculated ratio [K2O] / [NA2O]. For records where NA2O is zero the else statement sets the label to nothing (the empty double quotes "").


Image 1040



Disclaimer: although you can use expressions in most text/label field responses there are still some places in this Beta that don’t support expression labelling.


Expression Editor

If you’re wondering how to edit a complex multi-line expression in a (relatively) small field response, simply right-click it and choose Edit Expression from the pop-up menu. Micromine will display a formatted version of the expression with the = already included. The expression editor also lists the Available Fields and Available Functions, from which you can quickly build an expression by double-clicking fields and functions as required. For now you still have to manually enter everything else.


Image 1041



Clicking the Check button gives you an opportunity to validate and correct the expression before you use it.


Although it’s perfectly valid to enter a large expression on one line:


=if([K2O] > [NA2O]) then "Potassic" else "Sodic" endif


I think you’ll agree it’s much easier to read on multiple lines.


The same applies to whitespaces. You can build expressions with or without spaces, although using them does make your expressions easier to read. A simple rule of thumb is to add a space wherever necessary to improve clarity.


Filters

So far I’ve used expressions in colour fields and text fields, purely because they’re visual and easy to understand. But you can use expressions anywhere you’d use an input file field, and two obvious places are in a filter and in the calculator.

To reproduce the [K2O] > [NA2O] expression in a filter:


Image 1042



Please note that a filter is just an if statement; the then is whichever task has the filter applied. Other than reversing the filter there is no equivalent to an else statement.


Calculations

I mentioned at the beginning of this post that expressions are like equations, and of course they’re easily used as equations. In fact, we’ve made a whole new calculator that only uses expressions, which you can find at File | Fields | Calculate (Expression). This is where expressions really start to fly; instead of setting up a multi-stage calculation using the existing Calculator, simply write the equation as an expression.


Suppose you wanted to convert polar coordinates to rectangular coordinates. In the old Calculator you would do this:


Image 1044



In the new expression-based calculator it looks like this:


Image 1045



As you can see, each expression combines two calculations into one step without resorting to temporary variables, producing a much more succinct solution that is readily understood and easily maintained.


Disclaimer: we’re still working on the underlying expression mathematics and the current function list in this Beta is pretty limited.


As a working example, the limestone industry uses the lime saturation factor (LSF) to determine the characteristics of the raw material. The calculation looks like this:


= 100 * [CAO] / (2.8 * [SIO2] + 1.18 * [AL203] + 0.65 * [FE203])


I’ll leave this as an exercise for you to recreate using the old calculator.


Next/previous record

When you’re using Calculate (Expression) you’ll notice that the Available Fields list includes fieldnames with ‘<’ prepended and ‘>’ appended to each name. For example, [EAST] will be accompanied by [<EAST] and [EAST>]. You use the prepended (<) version to obtain that field’s value from the previous record, and the appended (>) version to obtain the field’s value from the next record.


Suppose you wanted to measure the differences between successive records in a non-drillhole file:


=[VALUE]-[<VALUE]


Now that was easy, wasn’t it?


Colour construction

Everyone seems to be acquiring LiDAR data nowadays, and many surveys produce point clouds where each point has R,G,B attributes in addition to the standard X,Y,Z values. In other words each point knows its colour.


A vexing problem of dealing with this data is to generate colour from a file that has separate red, green and blue value fields. Micromine 2014 introduced the ability to automatically display colour directly from a colour field, but its biggest limitation was that you had to manually convert the RGB triplets into a number Micromine could understand.


Nowadays you can simply use the RGB() function, like this:


=RGB([R],[G],[B])


where [R], [G], and [B] are the red, green and blue fieldnames.


Image 1046



Highlight/use every nth record

My final example is to display, label, or modify every nth record. This method that pretty much requires you to modify the input file, but it’s still far more efficient to use an expression to make the change.


A typical every nth scenario is to create contours from a DTM or grid surface and then label every nth contour as Index, with the remainder labelled as Intermediate. Suppose you generated two-metre contours and wanted multiples of 10 to be Index contours:


=if([ELEV] % 10) = 0 then
    "Index" 
else 
    "Intermediate" 
endif


The % (modulo or modulus) operator calculates the remainder after division. For example, 12 % 10 is 2. At school you might have said it as “12 divided by 10 is 1, with remainder 2”. Similarly, 20 % 10 is zero because there is no remainder.


In the contouring example, elevations that are multiples of 10 have zero remainder and are labelled as Index contours. Everything else has a non-zero remainder and is labelled as Intermediate. You can use this expression to create labels on-the-fly, or use it in Calculate (Expression) to produce a permanent label field.


Another every nth scenario is to simplify a calculation by only using every nth record. Here you must modify the file because this method relies on a record ID (RID) field, which is simply an incremental value that reproduces the row labels at left of the File Editor. To use every 25th record in a calculation or for labelling:


=if ([RID] % 25) = 0 then "Label this" else "" endif


Conclusion

I hope this introduction to expressions in Micromine 2016 Beta whets your appetite to experiment with them on your data, remembering that we’re still finalising the feature list and some minor things may change. Having said that, we’re pretty excited about the potential that expressions offer.


+1

UPDATE


Good news everyone (to quote a certain professor),


We've added [#record] as an expression variable that provides direct access to the record number of the current record. So there's no need to create a record ID field as per my last expression example.


The every nth scenario now looks like this:


=if ([#record] % 25) = 0 then "Label this" else "" endif


One potential trap for the unwary is that record IDs start at 1, which means the first record will not be chosen. If this is a problem, just subtract 1 from the record ID:


=if (([#record]-1) % 25) = 0 then "Label this" else "" endif


We've also added more mathematical operators and are reviewing candidates for yet others. Happy days indeed.

Hello Frank


I was trying to resolve issue which Evgeniya has described using Desurvey Utility and Comulative lenght. Then in configured point label from the field using Edit expression tool.


Then press okay and receive the following error


So as i understand MM doesn't recognize this value or MM worked till the end and didn't found the following value for current row.


Is there any way to find workaround or i'm just missed something?

+1

Hello Taras,


I'll have to check with our developers, but I think the problem is that the '>' version of the field name is not supported for labelling -- it only works in the expression calculator.


You can confirm this by opening the Expression Editor in Calculate (Expression) -- you'll see all of your file fields listed along with their '<' and '>' versions.


Also, without seeing your data I can't tell if your file includes points that are exactly 100 metres apart. The modulo operator won't find any data unless the differences exactly match the number you've supplied in the expression.


One way to overcome both problems is to use Condition String to "seed" the strings with points that are exactly 100 m apart. You can then use the expression calculator (which does support the '>' field version) to permanently write the labels to the file.


Perhaps I don't understand the scenario, but I would simply use the Vizex Contour form to set up the labels, and set 'Distance apart' on the 'Labels' tab to 100.


+1

This can be made to work if points are inserted into the file at exactly 100 metre intervals.


The easiest way to do that is by selecting all of the required strings in Vizex and then choosing Insert Points from the right-click menu (not Condition String as I had previously written). Be patient because this process takes a long time to process a large file.


Once that's done, run Strings | Utilities | Desurvey and enter a name for the Cumulative length field.


Then use a basic modulo (%) expression to isolate the labels falling on cumulative distance multiples of 100 metres:


#if(INT([CumLen]) % 100 = 0) then
    [RL]
 else
    ""
endif

The INT() is just there to force the remainders to be integers, because cumulative distance is a REAL field and some numbers can't be represented exactly, even with full precision.


You can use this expression in the Vizex String dialog to label points and segments.


Oh, and the Vizex Contour form doesn't work in this scenario because the "contours" don't have constant elevations.


Hi Frank


thanks for detailed explanation, it looks like the way you have described is the simpliest one for user.

+2

Just wanted to say thanks for this feature and highlight how this is another example where you guys make an improvement and make it available throughout the program.

This is MicroMine's best attribute; letting the used decide how to use the features.

For example, I needed to do a merge but the string in one file needed to be concatenated from two columns to match one column in another file, dynamic expressions enabled this.

Hi Geoff,


Thanks for your kind words and for sharing your experience with us. We couldn't possibly know how our users would take advantage of this feature, but your excellent example shows just how well they can help out in a whole bunch of different ways.


Thanks again for letting us know.


Frank


Frank,

Is it possible to add dynamic field expression support to improve the flexibility of the Box and Whisker Plot?

Maybe this is already possible and my equation isn't correct.

It would be handy to filter by zone as shown. This way you wouldn't require a different column for each domain.

Thanks, Geoff.


Hi Geoff,

I'm not sure if this will fix your problem but try adding spaces between the words. e.g

= if [ZONE] = 1 then [Au_GPT] else 0 endif

I think I also found an occasion when the expression worked when not using the square brackets around the field names but it wouldn't work with the square brackets so you could try that too.

Hope that helps

Rupert

Hi Geoff and Rupert,


We're progressively adding expression support to more parts of Micromine. In MM 2016 Service Pack 4 you can now do exactly what Geoff suggested in Stats | Quick Summary. I know this doesn't address Geoff's question, but it is at least a step in the right general direction.


As you've probably figured out, we make these changes on a feature-by-feature basis, and we haven't gotten to the more advanced stats tools yet. (And to confirm, I tested the Box-and-Whisker plot myself and got the same "no results" result.)


Frank


Great new addition!! So nice to have something I discussed with Frank sometime back get added to the functionality of Micromine (r,g,b colour compositing).

Keep up the effort guys!