handle a long running process nicely

[The Human Tangent]

I have a very long running server side process that imports items from an XML file. It needs to run in a simple web page. Importing 3 items takes about 25 seconds due to a very slow running API process and not much can be done about that. In total the process needs to import about 1800 items, so there is no way this can run before the page can complete without timing out. Also I need to give some kind of simple visual progress to the user (importing 37 of 1997 etc.)

I've put the process to import as a stand alone file that takes a query string (x= current row position ; y = total rows ; f = filename saved on server )

Originally I thought that the first request could call the second request etc.

So processXml.aspx ?x=1&y=1997&f=m yfile.xml when completed would reload processXml.aspx ?x=2&y=1997&f=myfil e.xml then processXml.aspx ?x=3&y=1997&f=myfil e.xml etc

however if i use

string fileName = Request.QueryString["f"].ToString();
string tX = Request.QueryString["x"].ToString();
string tY = Request.QueryString["y"].ToString();
int iX = Convert.ToInt16(tX);
string sX2 = Convert.ToString(iX+1);

///do a long running thing
...
Response.Redirect("processXml.aspx?x=" + sX2 + "&y=" + tY + "&f=" + fileName, true);

[sX2 = x+1]

no output is returned to the user

then I thought what about

Response.AppendHeader("Refresh", "0; processXml.aspx?x=" + sX2 + "&y=" + tY + "&f=" + fileName);

but this always loads the same page with the same query string, no ideas why, but X does not progress, it stays in a loop where x always equals 1, on page load after the reload x always equals 1, and sX2 always equals 2 but when reloaded x still equals 1

So....

what is a good approach to this?

I'm wondering about using an IFrame maybe and calling it in JavaScript but this would be new territory for me, there must be a good pattern for this sort of thing

Any help very gratefully received :)

[Frinavale]

It's pretty hard to achieve this because of the disconnected nature of the web environment that you're working with....

Think about what generally happens:

• the web browser makes a request to the server for a page
• the web server passes the request to your code and your code generates the page
• the page is sent to the browser
• the page is rendered by the browser

Now, your process is going to take a long time....so this means that the page cannot be sent to the browser until it's finished processing.

I have tried getting around this a few times but have run into a wall every time. What I've tried doing is preforming an Ajax request to the server to start the process and then have an Ajax Timer control postback to the server to see how far along the process is.

You can try to do this but I have a feeling that you're going to run into the same problem I did while trying to solve this problem.

I don't know how to get around the road block that I've run into. If you like to see how far I've gotten check out post #15 in this thread: ASP.NET Progress

If you figure out how to get around my problem please let me know!

Happy coding,
-Frinny

[The Human Tangent]

In the end the answer was very simple.

I've had a great read through the thread you suggested and some other suggested from there and so on, was greatly amused by the closed thread that had the helpful comment "JUST USE AJAX" yes in proper shouty-case :)

It's a great site this one and I am sure I'll keep coming back, thank you for the friendly helpful welcome.

In the end the answer was a very simple bit of javascript.

I broke the process into a single page that does one part and then calls the next page. Once the process is complete and if x<y (ie we're not on the last one) then I write a timed window relocate, but after everything else.

eg:

                Response.Write("<h2>Completed " + tX + " of " + tY + "</h2>");

                string sNextX = "processXml.aspx?x=" + sX2 + "&y=" + tY + "&f=" + fileName.Replace("\\", "\\\\");

                StringBuilder sbScript = new StringBuilder();
                sbScript.AppendLine("<script type=\"text/javascript\">");
                sbScript.AppendLine("setTimeout(\"GoNextX()\", 2000);");
                sbScript.AppendLine("function GoNextX()");
                sbScript.AppendLine("{");
                sbScript.Append("window.location = \"");
                sbScript.Append(sNextX);
                sbScript.AppendLine("\"; ");
                sbScript.AppendLine("}");
                sbScript.AppendLine("</script>");

                ClientScript.RegisterStartupScript(typeof(Page), "nextX", sbScript.ToString());

so at the top the address to the next call is processed - with a lot of quadruple slashing going on for the benefit of javascript and c#

after the process has completed the redirect is written to the browser and it processes the next one, but crucially the current page terminates in memory and flushes all its output to the page

this now runs as a 3hour+ request absolutely fine :)

[Frinavale]

I don't think I fully understand your application needs.

If you preform a Response.Redire ct() then the web browser is told by the server to redirect to another web page. I'm glad that you were able to get this to work for your needs...but I'm a little disappointed that my question still remains.

I've been thinking about it in my spare time (on lunch, while driving, etc) and you have given me a new idea on how to approach this problem.

I'm thinking that maybe I could use Server.Transfer to shift the request off to to be processed by another ASPX page, thus freeing the original page up for accepting new requests. (Recall that Server.Transfer sends all of the information that has been assembled for processing by one .asp file to a second .asp file...this means that I could gather all of the information necessary to do the processing in the original page and send all of that information to the next page which would do the process). I think that this might work! It seems sound in theory, but so does handling asynchronous requests to the same ASPX page asynchronously so I'm not 100% sure.

-Frinny