When Stubbing function call it throws cannot be cast to org.apache.spark.sql.Dataset java.lang.ClassCastException:

[shikhil-gupta]

Hi Team, I have written this test but I am facing issue.

test("applyNewDeviceEnrolledChanges") {

val dao: DataAccessLayerDAO = mock(classOf[DataAccessLayerDAO])

val metricStartTimeStamp : Timestamp = new Timestamp(LocalDate.parse("2021-10-03").atStartOfDay(ZoneId.of("UTC")).toInstant.toEpochMilli)

val metricEndTimeStamp : Timestamp = new Timestamp(LocalDate.parse("2021-10-04").atStartOfDay(ZoneId.of("UTC")).toInstant.toEpochMilli)

val rollingPeriod : Int = 1

val metaDataProcessor : DeviceMetadataProcessor = spy(new DeviceMetadataProcessor(spark, dao, metricStartTimeStamp, metricEndTimeStamp, rollingPeriod))

doAnswer(new Answer[DataFrame] {
override def answer(invocation: InvocationOnMock): DataFrame = {
spark.emptyDataFrame
}
}).when(metaDataProcessor).loadMasterTableOfNthDay()

metaDataProcessor.process()
}

**class DeviceMetadataProcessor(sparkSession: SparkSession, dataAccessDao: IDataAccessLayer, metricStartDate: Timestamp, metricEndDate: Timestamp, rollingPeriod: Int) extends IProcessor
{
var deviceMasterTableDataNthDay: DataFrame = null;
override def process(): Unit = {

	deviceMasterTableDataNthDay = loadMasterTableOfNthDay()
    }
def loadMasterTableOfNthDay(): DataFrame = {
	var masterTable : DataFrame = dataAccessDao.getDeviceProcessedMetaData(deviceMasterTableTemplatePath, metricStartDate, DateUtilities.getStartOfDayTimeStamp(metricEndDate, -1 * rollingPeriod), 1)
}

}**

While stubbing loadMasterTableOfNthDay function of DeviceMetadataProcessor and I am trying to return empty dataframe but it throws me following errors.

Exception:-

com.microsoft.teams.deviceanalytics.core.test.DeviceMetaDataProcessorTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anon$1 cannot be cast to org.apache.spark.sql.Dataset
java.lang.ClassCastException: com.microsoft.teams.deviceanalytics.core.test.DeviceMetaDataProcessorTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anon$1 cannot be cast to org.apache.spark.sql.Dataset
at com.microsoft.teams.deviceanalytics.core.processor.DeviceMetadataProcessor.process(DeviceMetadataProcessor.scala:50)
at com.microsoft.teams.deviceanalytics.core.test.DeviceMetaDataProcessorTest$$anonfun$1.apply$mcV$sp(DeviceMetaDataProcessorTest.scala:35)
at com.microsoft.teams.deviceanalytics.core.test.DeviceMetaDataProcessorTest$$anonfun$1.apply(DeviceMetaDataProcessorTest.scala:20)
at com.microsoft.teams.deviceanalytics.core.test.DeviceMetaDataProcessorTest$$anonfun$1.apply(DeviceMetaDataProcessorTest.scala:20)
at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:186)
at org.scalatest.TestSuite$class.withFixture(TestSuite.scala:196)
at org.scalatest.FunSuite.withFixture(FunSuite.scala:1560)
at org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:183)
at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:196)
at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:196)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:289)
at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:196)
at org.scalatest.FunSuite.runTest(FunSuite.scala:1560)
at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:229)
at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:229)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:396)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384)
at scala.collection.immutable.List.foreach(List.scala:392)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:379)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:461)
at org.scalatest.FunSuiteLike$class.runTests(FunSuiteLike.scala:229)
at org.scalatest.FunSuite.runTests(FunSuite.scala:1560)
at org.scalatest.Suite$class.run(Suite.scala:1147)
at org.scalatest.FunSuite.org$scalatest$FunSuiteLike$$super$run(FunSuite.scala:1560)
at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:233)
at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:233)

[shikhil-gupta]

@bbonanno Can you please look into this issue?

[ultrasecreth]

Given the syntax you're using, you don't seem to be using mockito-scala, would you mind refactoring that and try again?
also pls format the code in the issues, is really hard to follow otherwise

[ultrasecreth]

To build on top of my prev comment, for this kind of weird errors, I'd need a repo with some code that can be run and reproduce the issue