[ZEPPELIN-6546] Restore Math.max accumulation for NoteJobInfo.unixTimeLastRun#5319
Open
HwangRock wants to merge 2 commits into
Open
[ZEPPELIN-6546] Restore Math.max accumulation for NoteJobInfo.unixTimeLastRun#5319HwangRock wants to merge 2 commits into
HwangRock wants to merge 2 commits into
Conversation
…eLastRun The NoteJobInfo constructor overwrote lastRunningUnixTime on every loop iteration, so a note's last-run time always came from its last paragraph instead of its most recently run one. This breaks Job Manager sorting and the incremental LIST_UPDATE_NOTE_JOBS filter, which can silently drop a note that just ran. ZEPPELIN-2860 fixed exactly this with Math.max in 2017 (commit 4a369f1). The ZEPPELIN-3737 refactor (commit 001c621) moved the logic into JobManagerService and dropped the Math.max, silently reintroducing the bug. Restore the max accumulation.
Two tests build a note whose first paragraph finished recently while the last paragraph was created earlier and never ran, then pin the computed unixTimeLastRun through the getNoteJobInfoByUnixTime filter boundary: the note must be included for thresholds below the newest paragraph timestamp and excluded at exactly that timestamp. Both tests fail without the Math.max restoration.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is this PR for?
Job Manager reports a wrong "last run" time for any note whose most recently executed paragraph is not the last paragraph in the note.
NoteJobInfo(Note)inJobManagerServiceoverwriteslastRunningUnixTimeon every loop iteration, so the last paragraph always wins:Consequences:
unixTimeLastRun(jobmanager.filter.jsin zeppelin-web,job-manager.component.tsin zeppelin-web-angular), so the displayed relative time and the Recently-Update sort order are wrong.getNoteJobInfoByUnixTime()filters withunixTimeLastRun > lastUpdateServerUnixTime, so a note that just ran can be silently dropped fromLIST_UPDATE_NOTE_JOBSpushes when an older paragraph sits at the bottom.This is a regression of ZEPPELIN-2860, which fixed exactly this in 2017 with
Math.max(#2543). The ZEPPELIN-3737 refactor moved the logic from NotebookServer into JobManagerService and dropped theMath.max(001c621). This PR restores it.Verified on a live server: a two-paragraph note whose first paragraph finished today and whose second paragraph was created 8 days ago and never ran showed "8 days ago" in Job Manager before the fix and the correct run time after.
Screenshots (if appropriate)
2026-07-18.6.37.10.mov
What type of PR is it?
Bug Fix
Todos
Math.maxaccumulation inNoteJobInfoWhat is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-6546
How should this be tested?
mvn test -pl zeppelin-server -Dtest=JobManagerServiceTest— the two new tests pinunixTimeLastRunto the max paragraph timestamp via thegetNoteJobInfoByUnixTimefilter boundary and fail without the fix.Manually: create a note with two paragraphs, run only the first one, open the Job Manager page. Before the fix the note shows the second paragraph's creation date as its last run and sorts accordingly; after the fix it shows the actual run time.
Questions: