Speed up hudi e2e

[sarah-witt]

What does this PR do?

Speeds up hudi e2e; previously it would download spark which would could take over an hour and cause the test to time out. Now it takes ~2 minutes!

Motivation

speed up tests and reduce flakiness

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
• If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

[datadog-official]

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 73.91%

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 93fa550 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.99%. Comparing base (4e2d1f3) to head (93fa550).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[nubtron]

Nice! This was a big time-waster when re-running the whole test suite.