siddharth's space

Learn JShell

Java 9 ships with jshell which provides an interpretive shell-like environment (a REPL) for Java. This is extremely useful for prototyping, testing and learning how to use a new library. Here I describe how to use it in context of your Maven project. Simply run:

$ mvn com.github.johnpoth:jshell-maven-plugin:1.3:run

from the root directory of your Maven project. It will launch a new shell and import all the dependencies declared in your pom.xml! From here on you can start hacking away. Below I describe how I used jshell to test connection to MySQL db:

jshell> import java.sql.*;

jshell> String url="jdbc:mysql://x.y.z.w/my_db";
url ==> "jdbc:mysql://x.y.x.w/my_db"

jshell> var conn = DriverManager.getConnection(url, username, password);
conn ==> com.mysql.cj.jdbc.ConnectionImpl@279fedbd

jshell> conn.isValid(10);
$4 ==> true

jshell> /exit
|  Goodbye

Tips on using Reflection in Java

Problem: java.lang.NoSuchMethodException when calling getConstructor

Solution: check the class is marked as public

Problem: java.lang.SecurityException: Invalid signature file digest for Manifest main attributes when running a fat jar built using Maven shade plugin

Solution: Add following to pom.xml under the Shade plugin

<configuration>
            <filters>
            <!-- this filter will avoid Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes. see
        https://stackoverflow.com/a/6743609/147530 -->
        <filter>
            <artifact>*:*</artifact>
            <excludes>
                <exclude>META-INF/*.SF</exclude>
                <exclude>META-INF/*.DSA</exclude>
                <exclude>META-INF/*.RSA</exclude>
            </excludes>
        </filter>
    </filters>
...
</configuration>

Debugging Java programs from the command-line with jdb – Java command-line debugger

You use jdb for this. There is surprisingly little information available on this (esp. the commands). Start with official documentation [1]. Refer this for more.

Here are some of the most commonly used JDB commands

• run – Starts the execution of the program.
• stop at <class>:<line> – Sets a breakpoint at the specified line in the specified class.
• stop at <class>.<method> – Sets a breakpoint at the beginning of the specified method in the specified class.
• step – Executes the current line of code and stops at the next line. If the current line contains a method call, JDB steps into the method and stops at the first line of the method.
• stepi – step into
• next – Executes the current line of code and stops at the next line. If the current line contains a method call, JDB executes the method call and stops at the next line after the method call.
• cont – Continues the execution of the program until the next breakpoint or until the program completes.
• list – Lists the source code around the current execution point.
• print <expression> – Prints the value of the specified expression.
• locals – Lists the local variables in the current frame.
• classes – Lists all loaded classes.
• methods <class> – Lists all methods in the specified class.
• thread – Lists all threads and their current status.
• suspend – Suspends all threads.
• resume – Resumes some or all the threads. otherwise its similar to cont.
• help – Displays a list of available JDB commands.
• exit – Exits JDB.
• clear – delete breakpoint. clear class:line or clear class.method
• up/down – move up and down the call stack.
• where – prints the callstack
• dump – similar to print

> help
** command list **
connectors                -- list available connectors and transports in this VM

run [class [args]]        -- start execution of application's main class

threads [threadgroup]     -- list threads in threadgroup. Use current threadgroup if none specified.
thread <thread id>        -- set default thread
suspend [thread id(s)]    -- suspend threads (default: all)
resume [thread id(s)]     -- resume threads (default: all)
where [<thread id> | all] -- dump a thread's stack
wherei [<thread id> | all]-- dump a thread's stack, with pc info
up [n frames]             -- move up a thread's stack
down [n frames]           -- move down a thread's stack
kill <thread id> <expr>   -- kill a thread with the given exception object
interrupt <thread id>     -- interrupt a thread

print <expr>              -- print value of expression
dump <expr>               -- print all object information
eval <expr>               -- evaluate expression (same as print)
set <lvalue> = <expr>     -- assign new value to field/variable/array element
locals                    -- print all local variables in current stack frame

classes                   -- list currently known classes
class <class id>          -- show details of named class
methods <class id>        -- list a class's methods
fields <class id>         -- list a class's fields

threadgroups              -- list threadgroups
threadgroup <name>        -- set current threadgroup to <name>
threadgroup               -- set current threadgroup back to the top level threadgroup

stop [go|thread] [<thread_id>] <at|in> <location>
                          -- set a breakpoint
                          -- if no options are given, the current list of breakpoints is printed
                          -- if "go" is specified, immediately resume after stopping
                          -- if "thread" is specified, only suspend the thread we stop in
                          -- if neither "go" nor "thread" are specified, suspend all threads
                          -- if an integer <thread_id> is specified, only stop in the specified thread
                          -- "at" and "in" have the same meaning
                          -- <location> can either be a line number or a method:
                          --   <class_id>:<line_number>
                          --   <class_id>.<method>[(argument_type,...)]
clear <class id>.<method>[(argument_type,...)]
                          -- clear a breakpoint in a method
clear <class id>:<line>   -- clear a breakpoint at a line
clear                     -- list breakpoints
catch [uncaught|caught|all] <class id>|<class pattern>
                          -- break when specified exception occurs
ignore [uncaught|caught|all] <class id>|<class pattern>
                          -- cancel 'catch' for the specified exception
watch [access|all] <class id>.<field name>
                          -- watch access/modifications to a field
unwatch [access|all] <class id>.<field name>
                          -- discontinue watching access/modifications to a field
trace [go] methods [thread]
                          -- trace method entries and exits.
                          -- All threads are suspended unless 'go' is specified
trace [go] method exit | exits [thread]
                          -- trace the current method's exit, or all methods' exits
                          -- All threads are suspended unless 'go' is specified
untrace [methods]         -- stop tracing method entries and/or exits
step                      -- execute current line
step up                   -- execute until the current method returns to its caller
stepi                     -- execute current instruction
next                      -- step one line (step OVER calls)
cont                      -- continue execution from breakpoint

list [line number|method] -- print source code
use (or sourcepath) [source file path]
                          -- display or change the source path
exclude [<class pattern>, ... | "none"]
                          -- do not report step or method events for specified classes
classpath                 -- print classpath info from target VM

monitor <command>         -- execute command each time the program stops
monitor                   -- list monitors
unmonitor <monitor#>      -- delete a monitor
read <filename>           -- read and execute a command file

lock <expr>               -- print lock info for an object
threadlocks [thread id]   -- print lock info for a thread

pop                       -- pop the stack through and including the current frame
reenter                   -- same as pop, but current frame is reentered
redefine <class id> <class file name>
                          -- redefine the code for a class

disablegc <expr>          -- prevent garbage collection of an object
enablegc <expr>           -- permit garbage collection of an object

!!                        -- repeat last command
<n> <command>             -- repeat command n times
repeat                    -- show whether GDB-style empty command repetition is enabled
repeat <on|off>           -- enable/disable GDB-style repetition
# <command>               -- discard (no-op)
help (or ?)               -- list commands
dbgtrace [flag]           -- same as dbgtrace command line option
version                   -- print version information
exit (or quit)            -- exit debugger

Here is how to do things with jdb in Q&A format:

How do I run jdb without needing to attach to a process?

jdb -classpath $CLASSPATH -sourcepath $SOURCE_PATH $MAIN_CLASS $ARGS

This will start jdb and immediately pause waiting for you to give the run command to continue execution. You should set some breakpoints before you run run otherwise the program will just execute without ever breaking.

How to attach to a running process?

The other option is to be able to attach to a running java program. In order to do this the java program must be run with following options which will enable jdb to attach to the process:

java -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=*:5005 ...

If you get some cryptic zsh error try escaping the * using a backslash \ like this:

java -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=\*:5005 ...

How do I provide the classpath to jdb?

The -cp option for specifying classpath does not work with jdb. You have to use -classpath. Another reason why I hate Java [1].

How do I print the callstack?

use where

How do I set breakpoint?

stop at com.mycompany.app.MyClass:65

How do I step into a method?

I could not find a way to differentiate between step into, step over or step out. The step command advances execution to the next line whether it’s in the current stack frame or a called method. The next command advances execution to the next line in the current stack frame.

How to print a variable?

locals will print local variables. use print to print a variable

Can I list all the loaded classes?

Yes, using classes

Decompiling Java Code

You can use decompiler.com if you don’t want to install anything. Otherwise install jd-gui or the fernflower decompiler. For jd-gui:

wget https://github.com/java-decompiler/jd-gui/releases/download/v1.6.6/jd-gui-1.6.6.jar

Then run it like:

java -jar jd-gui-1.6.6.jar

It will open a easy to use GUI.

It works pretty well but if you want to decompile a whole bunch of code and generate .java files to open in VS Code etc. try fernflower:

wget https://www.jetbrains.com/intellij-repository/releases/com/jetbrains/intellij/java/java-decompiler-engine/241.14494.240/java-decompiler-engine-241.14494.240.jar

Example usage:

java -jar java-decompiler-engine-241.14494.240.jar -hes=0 -hdc=0 $SOURCE_DIR $DEST_DIR

fernflower is also more up to date. With jd-gui I got this error when trying to decompile a file:

java.lang.IllegalArgumentException: Unsupported class file major version 61

fernflower was able to handle it. For small files you can use the java decompiler that comes with JDK:

javap -c -l Program.class

Understanding JDB Internals – How does the Java Debugger work?

The best way to understand the internals of anything is to study its source code. Luckily, jdb‘s source code is available publicly though finding it is somewhat of a treasure hunt and you can’t view it in Chrome like you can do with a github repo. But look no further. Download jdk-8-macosx-x64-demos.zip file from Misc. Java SE Tools and Libraries Downloads page. Extract the zip file. Then extract examples.jar under demo/jpda (or you can extract src.zip). jdb documentation is available under demo/jpda/com/sun/tools/example/doc/jdb.html. From there we see the Java file corresponding to the jdb program is demo/jpda/com/sun/tools/example/debug/tty/TTY.java. You can even modify the source code and add additional features e.g., one problem I have faced using jdb is inspecting the contents of a large array. In my case I wanted to print the URLs of an array containing 197 java.net.URL. There is no easy way to do this in jdb.

Running (and debugging) JUnit tests from the command-line

Why? because 9 out of 10 times the test runner in VS Code cannot recognize the tests [1] and I eventually gave up trying to wrestle with it. So free yourself from being dependent on it.

JUnit4:

java \
-cp $CLASSPATH \
--add-opens java.base/sun.nio.ch=ALL-UNNAMED \
--add-opens java.base/java.nio=ALL-UNNAMED \
--add-opens java.base/java.util.concurrent=ALL-UNNAMED \
--add-opens java.base/java.util.concurrent.atomic=ALL-UNNAMED \
--add-opens java.base/java.util.concurrent.locks=ALL-UNNAMED \
--add-opens java.base/java.util=ALL-UNNAMED \
org.junit.runner.JUnitCore \
com.mycompany.app.BdbTest

make sure you include target/test-classes in the CLASSPATH

JUnit5:

1. Download the junit-console-standalone jar
2. Compile your code: mvn compile and mvn test-compile
3. Get the classpath: mvn dependency:build-classpath
4. Run your tests from command-line like below (the add-opens and add-modules are just shown for illustration):

# get the runner from https://repo1.maven.org/maven2/org/junit/platform/junit-platform-console-standalone/1.9.3/
JUNIT_RUNNER=/Users/xxx/bin/junit-platform-console-standalone-1.9.3.jar

# make sure you ran mvn test-compile. replace class and method name as appropriate.
# to debug simply add -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=\*:5005 to the java command
# the -c and -m flags act like a UNION. both are applied as filters and their results are UNIONed.
# the method name must start with a fully qualified class name followed by a '#' and then the method name, optionally followed by a parameter list enclosed in parentheses
java \
--enable-preview --add-modules jdk.incubator.vector -ea \
--add-opens java.base/java.lang=ALL-UNNAMED \
--add-opens java.base/java.lang.invoke=ALL-UNNAMED \
-cp $JUNIT_RUNNER:target/test-classes:target/classes:$CLASSPATH \
org.junit.platform.console.ConsoleLauncher \
-c com.example.Tests \
-m com.example.Tests#test1

Now you are not at mercy of VS code and can also debug easily if you want.

example To execute tests using maven surefire plugin:

mvn -Dtest=com.web.DatabaseTests#test1 -DargLine="-DJWT_SECRET_KEY=hahaha! -Djava.library.path=lib" test

Above will run the test1 inside class com.web.DatabaseTests. It also shows how to set system properties (JVM arguments) using -DargLine

JUnit5 Tips

Note that by default JUnit 5 creates a new instance of the test class for every test method. In a way this is good as it avoids any unintended side-effects. No test can interfere with another test. This default behavior can be overriden if desired by using the @TestInstance(Lifecycle.PER_CLASS) annotation.

See contents of jar file without unzipping (extracting) it

unzip -l HTTPClient-0.3-3.jar

Concurrency Tips – When should you make a class thread safe?

There are so many tips here that it is a book unto itself but one is worth covering which is addressing the question When should you make a class thread safe? First, we need to be clear on what we mean by thread safe. It does not need a big blog post. Thread safety simply means this

(Definition): if a class is thread safe then multiple threads can call methods on instance of the class simultaneously (concurrently) and the result will be the same as if the methods had been called one after another by a single thread.

Note that we cleverly left out clarifying the issue that if methods are called one after another then implicit in this statement is a hidden order of invocation. What is the order? The order is undefined and that is what race-condition alludes to. The class only guarantees the result will be as if the methods had been called one after another (in some order) but the exact order is indeterministic and not defined or part of the contract.

Now we address the question when should a class be made thread safe? First, realize it is trivial to make any class thread safe by just adding the synchronized keyword to all the methods of the class. The effect is the same as if the client had locked on the instance before invoking the method. i.e,

public synchronized void foo();

is exactly equal to:

synchronized(obj) {
   obj.foo();
}

Given this, should you ever make a class thread safe using the trivial approach? Why not let the caller do it? Its something the caller can do easily if they want to. That way the class is more general – a caller that will not be calling the class from multiple threads will not have to pay the penalty of synchronization (obtaining a lock).

Finally we come to the guideline from Effective Java (quoting from the book p. 322) which I think is on point:

If you are writing a mutable class, you have two options: you can omit all synchronization and allow the client to synchronize externally if concurrent use is desired, or you can synchronize internally, making the class thread-safe. You should choose the latter option only if you can achieve significantly higher concurrency with internal synchronization than you could by having the client lock the entire object externally.

Note on jar signing – what it protects and what it doesn’t

Jar signing does not protect you against someone modifying the binary and then signing it with their own signature. All jar signing does is that provide evidence that the jar was signed by so and so person. If there was some mechanism which would only let the jar execute if it was signed by you then only you are protected. Bob can create a new jar and sign it with their credentials. It does not prevent the hacked jar from executing. All it does is show that the jar is not signed by you. So main benefit of jar signing is to protect against a user downloading a malicious binary e.g., if Bob attempts to sell a hacked binary pretending that its original and coming from you then a user can verify the binary is not original and refuse to buy it. But nothing prevents Bob from running the hacked binary on his machine should he chose to do so.

Super Useful Maven Command to get the full classpath

mvn -q -DincludeScope=runtime dependency:build-classpath -Dmdep.outputFile=cp.txt

then use it like

java -cp "target/classes:$(cat cp.txt)" \
     com.mycompany.app.Program \
     arg1 arg2 ...

Further Reading

• Pluralsight
• Internals of Maven plugin
• Official Docs