Video and audio analysis in Java

I would like to do some audio and video analysis in Java.

In a bit more detail, I would like to identify the points in audio/video that have either been monotonous for quite some time or have drastically changed compared to some previous state.

If you want to look at it in a mathematical way, I can try to explain it like this:

Example:

You have an audio file. You should extract the waveform of that audio file. You could try to approximate that waveform with some simpler function, that can be expressed as a closed formula. Let's call that function f(t).

Now, to find out how your function behaves (is it increasing or decreasing) at some point or interval, I guess I could use the first derivative,f'(t). If I'd like even more information, I assume second derivative, f''(t) would also come in handy.

So, if we assume we can do that then I guess I'd have 1 piece of information about the audio.

However, if I'm not mistaken, audio files can also have spectrograms, so I'm unsure how they fall into all of this.

So, the real question goes here: Is there a way to do this in Java (efficiently)? I've been doing some digging and I've found MusicG, however, the last update date is July 2012, which leads me to believe this may be abandoned.

The second part refers to video files, but without their audio component.

This is where I'll have more questions, so I'm just gonna go and shoot them.

How do you identify points of change in "pace" in videos?

Here's an example:

Imagine the video shows car driver's point of view while he's driving on a long, straight road. Since the surroundings are mostly the same, the pace could be described as "not changing much". At one point, the road begins to curve but the driver, due to him falling asleep" is not following the road that precisely, so the surroundings start to change somewhat, and so does the pace. At the apex of that curve there is a tree, which grows bigger and bigger as the car is approaching it. Here, the POV (and the pace) is changing quite a lot, since the tree is getting bigger and bigger. In the end, the car crashes into a tree, all hell breaks loose, the car starts to roll uncontrollably, which indicates a really intense pace.

I'm assuming one way could be to do an image segmentation and somehow determine which portions of the frames are changing, and how big are those portions to try to determine pace, but I'd like additional input.

If anyone has had prior experience doing any sort of related work in Java, what approaches did you explore and/or use? One thing that immediately comes to my mind is JavaCV, but as I said, with my limited experience, I'm unsure what to actually try.