Sunday, December 11, 2016

Eclipse annotation based null analysis



The annotation based null analysis is a tool to add information to the Java type system, which particular references can be null, and which will never be.

Eclipse on its own, can already do flow analysis and show places where the sequential logic will lead to null pointer accesses.

But the annotation based null analysis makes the checking more tightly. This leads to more null checks and therfore to more "fail fast". The errors become obvious right away at those places where the first time a null enters a "shall not be null" field/parameter, not later when it is accessed.

@NonNull // from org.eclipse.jdt.annotation
private Listener listener;

A field marked with @NonNull must be initialized with a valid reference. Either directly at the field declaration or in the constructor. Eclipse will flag missing initialization with an error, even if done from within a method called from the ctor

@Nullable // from org.eclipse.jdt.annotation
private Listener listener;

With the added information, Eclipse can help in more depth in finding errors.

With "Eclipse External Annotation" external libraries can be annotated very easily. This is essential to work with this null analysis.

In the end, the contract between caller and callee is enhanced with the information which parameter and return values can be null. Before, this could only be communicated through documentation.

Preparation


In Eclipse, the project's compiler settings must enable like this.
Project Properties -> Java Compiler -> Warnings/Errors -> 
  • Null analysis: Check "Enable annotation-based null analysis" (a popup will do the next 2 for you)
  • Null analysis: "Null pointer access" : Error
  • Null analysis: "Potential null pointer access" : Error

For the "Eclipse External Annotations" a folder holding this information must be set. In my project we use the location castle.util/eea

Project Properties -> Java Build Path -> Libraries ->
  • JRE -> External Annotations: /castle.util/eea
  • Plugin Dependencies -> External Annotations: /castle.util/eea

@NonNullByDefault

It makes sense to have "everything" to be non null by default and just mark those places where you want to allow nullable.

This is done with the annotation @NonNullByDefault. It can be applied onto a class or a package. The later makes most sense. 
For this, have a package-info.java in all relevant packages, with content like this.

@org.eclipse.jdt.annotation.NonNullByDefault
package castle.util;


Migration

When you work with existing code, the amount of potential errors found might be very big. Most probably you cannot fix all at once.

When you have the project settings stored in version control, you should stay with null analysis disabled in version control, while you work with them enabled locally. Then when all errors are fixed, the project settings can be committed.

Applying Annotations

The following is general about annotation, so you can replace @NonNull and @Nullable.

Sequence with modifiers

@Override
@Nullable
public static Listener getListener(){
    // ...
}


If you use SonarLint it might check for the sequence of modifiers. So while the @Nullable is meant to be on the return value, it shall be placed where you have the annotations for the method.

Arrays

Array are 2 things at once:

private Listener[] listeners;

Here the array itself can be null and/or the elements.

This make the element nullable:

@Nullable
private Listener[] listeners;
private @Nullable Listener[] listeners2; // same as before

To make the array itself nullable:

private Listener @Nullable[] listeners = null; // OK

And finally to have the array and the elements nullable:

@Nullable
private Listener @Nullable[] listeners = null;


Qualified names


If you want to use explicit package names or refer to nested types, the annotation must be applied like this:

private castle.testenv.base.config.@Nullable IExu exu = null; // OK

Also parameters and generics types can have these annotation.


Dealing with external code

You write simple code, it pops up warnings and errors.









Your method has the implicit @NonNull for the return value. But the JRE StringBuilder class is not having annotation information. So Eclipse cannot tell, if this is probably a problem. Hence a warning.

To solve this, you need to have the sources attached to your external code. If you have the JDK installed, this is normally the case.

Now you can make the annotation:

  1. navigate to the method
  2. place cursor onto the parameter or return value'
  3. press Ctrl-1 and take the choice







This creates a file in the previously configured eea location, if not yet existing.
I configured "/castle.util/eea", so it creates the files "/castle.util/eea/java/lang/StringBuilder.eea".

From the source code of this StringBuilder toString() or the documentation, i can make a judgement, if this return type is @NonNull or @Nullable.

If I would choose @Nullable, as a consequence, the warning in my code would turn into an error. Because now Eclipse knows that this is a problem.
If I would choose @NonNull, the warning in my code goes away.

The stored information in the eea file looks like this:

class java/lang/StringBuilder
toString
 ()Ljava/lang/String;
 ()L1java/lang/String;

It identifies the method with its signature and gives the alternative with the null annotation change. A 1 indicates @NonNull, a 0 the @Nullable.
Here you can read it as: the toString method with no argument and a return type of @NonNull String.

Overriding external methods, implementing interfaces


Example overriding equals. Due to @NonNullByDefault, the Object parameter is taken as @NonNull, but this is a conflict of nullable parameter of the external Object equals method.
If you class extends Object, for a foreign caller, it must be possible to call equals() with the same condition as usual. The caller might not know, that this is your class, and that you don't accept null.







To solve this, you need to make this method compatible.








In other cases, it might be clear that the API is not allowing for nullable parameter.
Then you can choose to annotate it as shown above.

Checking for null


Now let's look at code with nullables.

Checking nullable fields


@Nullable
private Path outputPath;

void action(){
    if( outputPath == null ){
        outputPath = getDefault();
    }
    // Potential null pointer access: this expression has a '@Nullable' type
    outputPath.toString();
}

Eclipse gives an error, even if we checked for null. The flow analysis in fact did see, that we assigned the field, but the problem is, that concurrent running code, could modify the field again to be null. 

The save solution is to read the field once into a local variable and write it back when needed.


void action(){
    Path path = outputPath;
    if( path == null ){
        path = getDefault();
        outputPath = path;
    }
    path.toString();
}



Now the flow analysis in Eclipse detects that in the last line, the path variable is always non-null.

Having utilities

I use utility methods. What i found useful for null analysis are these:

General null checking

class Assert {
    public static <T> @NonNull T isNotNull( @Nullable T obj )
}

If the parameter is failing the check, a RuntimeException is thrown, otherwise the value is returned, now as a @NonNull type.

Objects

Utils that can work with null.

class ObjectUtil {
    public static isEqual( Object o1, Object o2 )
    public static <T> T nonNullOrElse(@Nullable T vv, T el) {
    public static <T> T nonNullOrCompute(@Nullable T vv, Supplier<T> supplier)
}

Strings


class StringUtils {
    public String ensureNonNull( @Nullable String str )
    public static boolean hasContent(@Nullable String str)
    public static String toString(@Nullable Object o) {
    public static String hasContentOrElse(String str, String elseValue )
    public static String hasContentOrCompute(String str, Supplier<String> elseValue )
    public static boolean equalsIgnoreCase(@Nullable String str1, @Nullable String str2)
    public static boolean startsWith(@Nullable String string, String prefix)
    // ...
}

How to avoid nullables


For sure it would be all very easy if there would be no null at in the program.
But it is hard, because some things live shorter than our current object and still we need to reference them.

If there are many references to nullable fields, we need many places where we check for non-null, just to make the tool (Eclipse) happy. This leads to frustration.

One of the simple cases is, where we can apply good defaults. 
But you need to verify that the null state of the field is nowhere tested and a decision is made with it.
E.g. you have a "title" string in a dialog, you might be able to initialize it to the empty String and change the field to a nonnull.

The Java8 Optional class and the "Null object pattern" can be used, to turn nullable fields into nonnull ones. But this hides the problem. In fact you would work against the static analysis and turn all accesses into runtime behavior again.

In my opinion, i is the best case, when you can redesign the classes to have only few nullable fields.

As an example i want to show how to make a JFace dialog with only one nullable field.
The problem here is, that the initialization of all the widgets in the dialog is not done in the constructor, it is done in an overridden method called "createDialogArea()". 
The SWT widgets cannot be created upfront, because they require their parents, which do not yet exist be passed into their constructors.
But when i need access to those widget later, i need to have nullable references.

class MyDialog extends TitleAreaDialog {
    @Nullable Button enableCheckBox;
    public void createDialogArea( Composite parent ){
        // ...
        enableCheckBox = new Button( parent, SWT.CHECK );
        // ...
    }
}

With this, i have to check for non-null before all accesses for that widget. I might have many of those, and my code might became very ugly.

class MyDialog extends TitleAreaDialog {
    @Nullable Gui gui;
    public void createDialogArea( Composite parent ){
        gui = new Gui( parent );
    }
    class Gui {
        Button enableCheckBox;
        Gui( Composite parent ){
            // ...
            enableCheckBox = new Button( parent, SWT.CHECK );
            // ...
        }
    }
}

Now the code of the class is split in two parts, the outer and that in the inner. The inner class can have all the references as nonnull. I can register listeners calling into the Gui class and use other fields without checks.
When i override other methods in the main class, i need to forward to the Gui with a single null check only.






























Friday, October 21, 2016

The Channel Bundle: multiplexing for serial connections

In my applications, i often have the need to communicate over TCP and somehow share the connection for different purposes.

So I created the general purpose library Chabu to implement this.

Chabu implements the multiplexing over individual channels to go over a single one.



The main advantage is the flow control for each individual channel.
This means, in a situation where the receiver of one channel is blocked, the other channels can still work.

This is the same, as if you would have several TCP connections. But in my applications, i need to avoid having multiple connections and stick to a single one.
Reasons are:
- limited amount of free ports due to firewall rules
- memory needed for a TCP connection on embedded devices
- better utilization of the payload in the TCP connection, hence fewer interrupts
- encapsulate the internal needed channels, no need to be visible on the network.

Chabu is available for plain C, |Java and C# (.NET).
https://github.com/frankbenoit/Chabu




Sunday, November 15, 2015

GEF4 Tutorial - Part 7 - Add / Remove

In step 7 of this tutorial, the nodes can be added and removed.
Adding is done via mouse right-click. A palette is opened (with one item only) that will be added when pressed.
Removed of nodes can be done in two ways. Either hover over the node, a handle tool with red cross is shown. Or select the node and press DEL key.

For the source of this tutorial step see github - gef4.mvc.tutorial7.

Note: parts of this tutorial are copied from other examples or from forum postings.


Creating a palette on right-click

The new palette to add items, shall show up, when clicked on the canvas. For this, we bind a policy to the root part adapter map.

adapterMapBinder
    .addBinding(
        AdapterKey.get(FXClickDragTool.CLICK_TOOL_POLICY_KEY, "FXCreationMenuOnClick"))
    .to(FXCreationMenuOnClickPolicy.class);

Then we need to test if the click is really on this canvas. And it is needed to store the coordinates in 2 ways
1. in screen coordinates, to position the palette as popup.
2. in the canvas coordinates, which might be even different in scaling, if the viewer is currently zoomed.

FXCreationMenuOnClickPolicy extends AbstractFXOnClickPolicy. The click() method looks like this:

@Override
public void click(MouseEvent e) {
    // open menu on right click
    if (MouseButton.SECONDARY.equals(e.getButton())) {
        initialMousePositionInScreen = new Point2D(e.getScreenX(), e.getScreenY());
        // use the viewer to transform into local coordinates
        // this works even if the viewer is scrolled and/or zoomed.
        InfiniteCanvas infiniteCanvas = getViewer().getCanvas();
        initialMousePositionInScene = infiniteCanvas.getContentGroup().screenToLocal(initialMousePositionInScreen);
        // only open if the even was on the visible canvas
        if( infiniteCanvas.getBoundsInLocal().contains(initialMousePositionInScene) ){
            openMenu(e);
        }
    }
}

As palette, a Java FX popup is used. The image is set as graphics into a button instance in the popup. So we automatically have some hover behavior from the button. 

The popup is set to have a border line and a shadow, to look like a context menu.



Adding a new TextNode

The action now is again done via an operation, so it can be undone.

GEF4 provides a ready to use policy for this, the CreationPolicy.

CreationPolicy<Node> creationPolicy = root.getAdapter(CreationPolicy.class);
creationPolicy.init();
creationPolicy.create(
    textNode,
    contentPartModel,
    HashMultimap.create());

The textNode is the newly create object for the model. The contentPartModel is the ModelPart instance, that shall add the textNode.

The CreatePolicy delegates the work to the ModelPart#doAddContentChild(), which must be overridden, otherwise the base class implementation will throw an exception.

Deleting via short cut

Deletion of parts is just a matter of registering the TextNodePart:

adapterMapBinder.addBinding(AdapterKey.get(FXTypeTool.TOOL_POLICY_KEY))
    .to(FXDeleteSelectedOnTypePolicy.class);

It delegates the model changes to the parents part doAddContentChild() and doRemoveContentChild().

Delete with the handle tool

This implementation is completely taken from the Logo example.

This deletion is implemented as an icon getting visible when hovering over the TextNodePart.





HoverHandleRootPart is a parent for optionally more than one such handle action. Now we have only one action, the deletion, but there may be more to add. So HoverHandleRootPart manages the position at the upper left corner of the TextNodePart, and adds child handle parts in a VBox vertically downwards.

HandlePartFactory creates the handle parts. It is registered in the overriden bindIHandlePartFactory() of the GuiceModule. When requested to give the handle parts for TextNodePart, it constructs the HoverHandleRootPart with one child, the DeleteHoverHandlePart.



The DeleteHoverHandlePart gets a policy DeleteOnClickPolicy attached, that implements the action to be done. In this case, it delegates to the DeletionPolicy, which is part of GEF4.








Saturday, November 7, 2015

GEF4 Tutorial - Part 6 - Undo / Redo

In step 6 of this tutorial, the changes that can be done to the text nodes are undoable and redoable.

For the source of this tutorial step see github - gef4.mvc.tutorial6.

Note: parts of this tutorial are copied from other examples or from forum postings.

At the moment there are 2 kind of changes that can be done to TextNodes.
1. Drag the node and change its location.
2. Edit the text

To be able to make use of the undo/redo mechanism that Eclipse implements.
To read more about this, you can start here:


Every action that applies a change that shall be undoable, must be represented by an implementation of IUndoableOperation.

Example is the text change.
Before it looked like this, just changing the text in the editModeEnd().

getContent().setText(newText);

Now an operation is needed, that look like this:

public class ChangeTextNodeTextOperation extends AbstractOperation implements ITransactionalOperation {

    TextNodePart part;
    private String oldText;
    private String newText;
    public ChangeTextNodeTextOperation(TextNodePart part, String oldText, String newText ) {
        super( "Change Text in TextNode" );
        this.part = part;
        this.oldText = oldText;
        this.newText = newText;
    }

    @Override
    public IStatus execute(IProgressMonitor monitor, IAdaptable info) throws ExecutionException {
        part.getContent().setText(newText);
        return Status.OK_STATUS;
    }

    @Override
    public IStatus redo(IProgressMonitor monitor, IAdaptable info) throws ExecutionException {
        execute(monitor, info);
        return Status.OK_STATUS;
    }

    @Override
    public IStatus undo(IProgressMonitor monitor, IAdaptable info) throws ExecutionException {
        part.getContent().setText(oldText);
        return Status.OK_STATUS;
    }

    @Override
    public boolean isNoOp() {
        return oldText.equals(newText);
    }
}

1. Get all information at creation time, because several operation might be executed, it is not reliable in which state the other objects are at that time.
2. Store as well the old state, so you can implement the undo.
3. If possible implement the redo with delegating to execute
4. Implement the isNoOp, so the operation with no effect are sorted out of the undo history.

To execute this operation you code like this:

getViewer().getDomain().execute(
    new ChangeTextNodeTextOperation(this, getContent().getText(), newText));

The Viewer has a reference to the domain. 
The domain's execute() method, delegates to the operation history. 
In case of working in a Eclipse plugin, the operation history is bound to the one from Eclipse, org.eclipse.gef4.mvc.ui.MvcUiModule. If you want to bind your own, you can overwrite the bindIOperationHistory in the Guice module.

The same pattern is applied to implement the ChangeTextNodePositionOperation.



Keyboard shortcuts

To have the keyboard shortcuts work for undo Ctrl-Z and redo Ctrl-Y, again a Policy is installed. But on which object. The keyboard input is send to the part which is selected, or the RootPart.
Guice can be used to add the policy to all TextNodeParts and the RootPart, but using only a single instance.

binder()
    .bind(GlobalOnTypePolicy.class )
    .in(Scopes.SINGLETON);

Configure the RootPartAdapters:

@Override
protected void bindAbstractRootPartAdapters(MapBinder<AdapterKey<?>, Object> adapterMapBinder) {
    super.bindAbstractRootPartAdapters(adapterMapBinder);
    adapterMapBinder
        .addBinding( AdapterKey.get( FXTypeTool.TOOL_POLICY_KEY, "GlobalOnTypePolicy"))
        .to( GlobalOnTypePolicy.class);
}

And then add the same mapping to the TextNodePart adapters.




GEF4 Tutorial - Part 5 - Editing text

In step 5 of this tutorial, the text nodes can be edited in the UI directly.

For the source of this tutorial step see github - gef4.mvc.tutorial5.


Note: parts of this tutorial are copied from other examples or from forum postings.

The text shall be editable when double clicked, or if the node is selected and ENTER is hit. The text is single line only. 
The text if ENTER is hit while editing. In case of ESC or if the focus is changed the edited test is discarded.













After the text is accepted the node size is adjusted, by keeping the position (left upper corner) stable.

To implement this the visual of the node is changed to use the StackPane as top level, having a RoundedRectangle, Text and TextField. The StackPane adjusts all widgets to be placed around the same center point. By switching on of the visibility and the managed state, it can show only the Text (normal mode) or the TestField (editing mode).

For this behavior, it is needed to react on mouse double click and on keyboard typing.
The classes TextNodeOnDoubleClickPolicy and TextNodeOnTypePolicy are added to the adapters. The given string for the role, is added to make those adapter not replace existing ones. So the value of the strings is in this case not important, it just must be different to what might be used elsewhere.

adapterMapBinder
    .addBinding( AdapterKey.get(
        FXClickDragTool.CLICK_TOOL_POLICY_KEY,
        "TextNodeOnDoubleClickPolicy"))
    .to( TextNodeOnDoubleClickPolicy.class);

adapterMapBinder
    .addBinding( AdapterKey.get(
        FXTypeTool.TOOL_POLICY_KEY,
        "TextNodeOnTypePolicy"))
    .to( TextNodeOnTypePolicy.class);

The double click policy checks if the click count is 2 and it was the primary button, then call to the part (TextNodePart) the new editModeStart() methods.

The type policy handles more cases in the «pressed» method:
F2 : start editing
ESC : cancel editing
ENTER if not yet in editing: start editing
ENTER if in editing: stop editing and take the value to the model

In case the key event was detected as action, the «event.consume()» shall be called, otherwise I had the case where the event was reported a second time.

The last case to handle is when the user clicks on another node and changes the focus.
For this, the TextNodePart registers/unregisters a listener for the focus model in it's doActivate/doDeactivate methods like this:

getViewer()
    .getAdapter(FocusModel.class)
    .addPropertyChangeListener( this::handleFocusModelUpdate );

If a change is detected and the current selected part is not this TextNodePart, the editing is cancelled.