Update: Managed to complete this task successfully now.
Observations: I was using the SCP plugin to push the build artifacts to ststor01:/data earlier. This plugin seems to be unstable as this time the plugin failed to upload the files in some iterations whereas it could in others. Could not investigate why as it seemed random. The SCP Plugin also gives security warnings and therefore it seems best to avoid it.
Instead this time, i used the SSH plugin to upload the files and it worked like a charm.
Thanks Admins @Inderpreet . Overall a good challenging task.