add new helper to remove duplicate struct in a slice.

implement the logic to prevent duplication when duplicate order on different file processed at same batch.