sequential command processing with an async io cloud upsert

by Ray Trask   Last Updated July 23, 2018 18:05 PM

We are new to c# and still trying to grok the async idioms.

We have a windows service that requires us to iterate a list of results queried from a PC database to feed the parse cloud server (which only supports async reads and writes). The results often have to be overridden with new results.

In a legacy java sdk, we simply did:

For each result:

do synchronous find if not found create a new object/document else Update the existing object/document Save synchronously and had no problems.

BTW - our server needs to feed Parse in seconds, not microseconds, though sub second performance would not hurt!

Our solutions to date seem to have two negative symptoms:
1) Duplication seems to occur when the command reading loop re-reads an “unprocessed command” that is currently processing and duplicates the write to the cloud. 2) after addressing 1 with locks surrounding our upsert cloud operation, we seem to be getting occasional deadlocks when running the service throughout the day.

Everywhere we look says, “DO NOT BLOCK or you will get deadlocks” and says go async/await all the way. But if we don’t block in some way we will get race conditions and duplicates. So we added locks and we get what seem like deadlocks after hours of processing commands just fine. A telling fact is that restarting the service processes the command queue just fine!!

This seems like a pretty common use case.

1) Is there an c# idiom we are missing here that loops through a command list to write asynchronously to the cloud but in sequential order without sending duplicates and not creating deadlocks?
2) How can we test this in such a way to prove it works in varying environments.

Note: We've looked at the following resources and are looking for direction so far.

Concurancy in C# Cookbook by Stephan Cleary

Async/Await Best Practices

Don't Block on Async Code

Related Questions

Need Async/Await equivalent for BackgroundWorker

Updated June 25, 2016 08:02 AM

An overview of how Asynchronous IO works in C

Updated August 08, 2018 06:05 AM