Iceberg class loading fails with ClassNotFoundException when Comet JAR is on system classpath

[karuppayya]

Describe the bug

Comet uses Class.forName(className) to reflectively load Iceberg classes. The single-argument Class.forName uses the classloader of the calling class. When the Comet JAR is placed in Spark's jars/ folder (system classloader) and Iceberg classes are bundled in the user's application uber JAR, these calls fail with ClassNotFoundException because parent classloaders cannot see classes loaded by child classloaders.

Steps to reproduce

1. Place the Comet JAR in Spark's jars/ folder
2. Bundle Iceberg classes in the application uber JAR
3. Submit the application — Iceberg scan operations fail with ClassNotFoundException

Expected behavior

Comet should be able to load the Iceberg classes in all cases

Additional context

No response