Skip to content

[OPS] SL1_RBT short duration product generate many error on production chain #1091

Description

@suberti-ads

Environment:

  • Delivery tag:
  • Platform: OPS Orange Cloud
  • Configuration:
    rs-addon-s3-sy2-stc: 1.15.0
    rs-addon-s3-sl1-nrt: 1.15.0
    rs-addon-s3-pug-nrt: 1.15.0

Traceability:

Current Behavior:
Some SY2 products failed to be ingested by metatda-catalog.

Metadata extraction failed: java.lang.NumberFormatException: For input string: "nan" at java.base/jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2054) at java.base/jdk.internal.math.FloatingDecimal.parseDouble(FloatingDecimal.java:110) at java.base/java.lang.Double.parseDouble(Double.java:543) at esa.s1pdgs.cpoc.metadata.extraction.service.extraction.files.ExtractMetadata.transformFromOpengis(ExtractMetadata.java:889) at esa.s1pdgs.cpoc.metadata.extraction.service.extraction.files.ExtractMetadata.processS3Coordinates(ExtractMetadata.java:861) at esa.s1pdgs.cpoc.metadata.extraction.service.extraction.files.ExtractMetadata.processProductXFDUFile(ExtractMetadata.java:585) at esa.s1pdgs.cpoc.metadata.extraction.service.extraction.files.MetadataBuilder.buildS3LevelProductFileMetadata(MetadataBuilder.java:243) at esa.s1pdgs.cpoc.metadata.extraction.service.extraction.S3LevelProductMetadataExtractor.extract(S3LevelProductMetadataExtractor.java:33) at esa.s1pdgs.cpoc.metadata.extraction.service.ExtractionService.handleMessage(ExtractionService.java:136) at esa.s1pdgs.cpoc.metadata.extraction.service.ExtractionService.apply(ExtractionService.java:105) at esa.s1pdgs.cpoc.metadata.extraction.service.ExtractionService.apply(ExtractionService.java:57) at org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.invokeFunctionAndEnrichResultIfNecessary(SimpleFunctionRegistry.java:897) at org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.invokeFunction(SimpleFunctionRegistry.java:853) at org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.doApply(SimpleFunctionRegistry.java:708) at org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.apply(SimpleFunctionRegistry.java:551) at org.springframework.cloud.stream.function.PartitionAwareFunctionWrapper.apply(PartitionAwareFunctionWrapper.java:84) at org.springframework.cloud.stream.function.FunctionConfiguration$FunctionWrapper.apply(FunctionConfiguration.java:754) at org.springframework.cloud.stream.function.FunctionConfiguration$FunctionToDestinationBinder$1.handleMessageInternal(FunctionConfiguration.java:586) at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:56) at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115) at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:133) at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106) at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:72) at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:317) at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:272) at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187) at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166) at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47) at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109) at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:216) at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.sendMessageIfAny(KafkaMessageDrivenChannelAdapter.java:397) at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.access$300(KafkaMessageDrivenChannelAdapter.java:83) at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:454) at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:428) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeOnMessage(KafkaMessageListenerContainer.java:2629) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeOnMessage(KafkaMessageListenerContainer.java:2609) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeRecordListener(KafkaMessageListenerContainer.java:2536) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeWithRecords(KafkaMessageListenerContainer.java:2427) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:2305) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:1979) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeIfHaveRecords(KafkaMessageListenerContainer.java:1364) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:1355) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:1247) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.lang.Thread.run(Thread.java:829)

It appears the sensing duration for these product is very short (less than one second).

Expected Behavior:
Product should not be generated in case of too short duration

Steps To Reproduce:
Sample processing

Test execution artefacts (i.e. logs, screenshots…)
see bellow

Whenever possible, first analysis of the root cause
It seems issue was due to SY2 rs-addon should not generate these products.
Log Preparation worker
On execution generate :
S3A_SY_1_MISR___20240703T142014_20240703T142014_20240703T183453_0000_114_153______LN3_D_ST_002.SEN3
S3A_SY_2_SYN____20240703T142014_20240703T142014_20240703T183504_0000_114_153______LN3_D_ST_002.SEN3
S3A_SY_2_VGP____20240703T142014_20240703T142014_20240703T183506_0000_114_153______LN3_D_ST_002.SEN3
S3A_SY_2_VGK____20240703T142014_20240703T142014_20240703T183509_0000_114_153______LN3_D_ST_002.SEN3
All these product have a short duration.
hereafeter job associated:
appjob151730.txt

Hereafter log from preparation worker
Explore-logs-2024-07-12 13_54_52.txt

Input for SY2-STC job come from PUG-NRT which have also issue due to low duration procut.
All these products seems to be generated by SL1-RBT Products.
Input product: S3A_SL_1_RBT____20240703T142014_20240703T142514_20240703T171153_0299_114_153______LN3_O_NR_002.SEN3

Hereafter job for preparation-worker for Sl1-NRT
appjob151442.txt

Hereafter preparation log for job generation:
Explore-logs-2024-07-12 14_53_53.txt

Bug Generic Definition of Ready (DoR)

  • The affect version in which the bug has been found is mentioned
  • The context and environment of the bug is detailed
  • The description of the bug is clear and unambiguous
  • The procedure (steps) to reproduce the bug is clearly detailed
  • The tested User Story / features is linked to the bug if available
  • Logs are attached if available
  • A data set attached if available

Bug Generic Definition of Done (DoD)

  • the modification implemented (the solution to fix the bug) is described in the bug.
  • Unit tests & Continuous integration performed - Test results available - Structural Test coverage reported by SONAR
  • Code committed in GIT with right tag or Analysis/Trade Off documentation up-to-date in reference-system-documentation repository
  • Code is compliant with coding rules (SONAR Report as evidence)
  • Acceptance criteria of the related User story are checked and Passed

Metadata

Metadata

Assignees

No one assigned

    Labels

    CCBIssue for CCBbugSomething isn't workingopsTicket from ADS operation team

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions