← Problems92. Carry the last sensor reading forwardMediumPySpark
00:00 / 20:00

Carry the last sensor reading forward

MediumNot attempted yetAsked at Deloitte, TCS

A sensor reports every hour, except when it does not. Downstream wants a reading on every row, so a gap takes the last value the sensor actually sent.

Input

readings sensor_id bigint reading_ts string -- ISO-8601 UTC, 'YYYY-MM-DDTHH:MM:SSZ' reading_c double -- null where the sensor reported nothing

Rows do not arrive in timestamp order.

Output

One row for every input row, same sensor_id and reading_ts, with:

  • reading_c — the reading if there is one, otherwise the most recent non-null reading from the same sensor at an earlier timestamp, and null if that sensor has not reported yet

Example

sensor_idreading_tsreading_c
12024-01-01T00:00:00Z10.0
12024-01-01T01:00:00Z10.0
12024-01-01T02:00:00Z10.0
12024-01-01T03:00:00Z12.5

Constraints

  • Forward only. A later reading never fills an earlier gap — that would put a value into a row that could not have known it.
  • Sensors are independent. One sensor never fills another's gap.
  • Leading nulls stay null. There is nothing to carry forward yet, and inventing a zero there is a reading that was never taken.
  • Row order is not graded. Column names and their order are.

Example 1a two-hour gap

Input · readings
reading_csensor_idreading_ts
1012024-01-01T00:00:00Z
null12024-01-01T01:00:00Z
null12024-01-01T02:00:00Z
12.512024-01-01T03:00:00Z
Output
sensor_idreading_tsreading_c
12024-01-01T00:00:00Z10
12024-01-01T01:00:00Z10
12024-01-01T02:00:00Z10
12024-01-01T03:00:00Z12.5

Example 2nothing to fill from yet

Input · readings
reading_csensor_idreading_ts
null12024-01-01T00:00:00Z
null12024-01-01T01:00:00Z
512024-01-01T02:00:00Z
Output
sensor_idreading_tsreading_c
12024-01-01T00:00:00Znull
12024-01-01T01:00:00Znull
12024-01-01T02:00:00Z5

Topics

window functionsdata qualitytime intelligence

Similar problems

Community-reported interview topic. Not an official company question and no affiliation is implied.

solution.py
Loading editor…
Draft not saved yet · Spaces 4 · UTF-8⌘↵ run ⌘⇧↵ submit
Nothing run yet

Run against the public tests, or submit to score against all of them.