CSV merge data via HTTP: redundant downloads?

Greets,

I have written an app that programmatically connects a mail merge [WordXML]
document to a source of CSV data by altering the XML to specify a URL as the
data source, and adding the appropriate field definitions. (The UI won't
let you do this, even though it offers .ASP files as a possible data
source -- go figure.)

This all works nicely, but there's one little quirk: when the user opens the
document, Word downloads the data between 3 and 4 times -- I watched it
happen using WireShark, it sends the same request to the server (thus
downloads the same data) multiple times.

As a matter of course this sort of thing is set to expire immediately, but
in this case I set it to expire 5 minutes in the future, hoping to leverage
browser cache -- but no love! Apparently Word's HTTP request bypasses
cache. (I verified that the headers returned by the server permitted it to
be cached, again using WireShark; the request is definitely cachable.)

A rational reason for this behavior exceeds the limits of my imagination!
You'd think that if there was even a chance of redundant downloads, they
would at least respect the cache control headers sent by the server, but
nooooo. Cache would've made it barely noticable...

Anyways, the problem isn't a show-stopper, but the needless waste of server
resources (oh yeah, and end-users' time) is always annoying... hoping maybe
someone here has some insight.


TIA,
Mark McGinty

I can't answer your question, but could not get to the point where Word
would open my csv from an http address so could not look at the traffic.

Maybe you could let me know precisely what changes you ended up making
in your xml. Here, I changed a local drive file:///c:\a\csv.csv address
to http://www.somewhere.com/csv.csv in two places:
a. a relationship element with
Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/mailMergeSource"
b. the SELECT statement in the w:query element

FWIW I don't look at the protocol side much so cannot guarantee to
interpret what I see correctly, but I do remember having a conversation
some years ago about another aspect of the way that Word dealt with http
addresses etc., where Word also seemed to insist on a pretty laborious
process (I think it was to follow a HYPERLINK field).

In this case, I notice that when I open the document, Word first tries
to issue a Webdav(?) PROPFIND, fails, then

issues a GET and receives a 305 Not Modified,
issues a HEAD and a GET and receives a 305 Not Modified
issues a HEAD and a GET and receives a 305 Not Modified
issues a HEAD

Peter Jamieson

http://tips.pjmsn.me.uk

On 21/02/2010 14:47, Mark McGinty wrote:
Greets,

I have written an app that programmatically connects a mail merge [WordXML]
document to a source of CSV data by altering the XML to specify a URL as the
data source, and adding the appropriate field definitions. (The UI won't
let you do this, even though it offers .ASP files as a possible data
source -- go figure.)

This all works nicely, but there's one little quirk: when the user opens the
document, Word downloads the data between 3 and 4 times -- I watched it
happen using WireShark, it sends the same request to the server (thus
downloads the same data) multiple times.

As a matter of course this sort of thing is set to expire immediately, but
in this case I set it to expire 5 minutes in the future, hoping to leverage
browser cache -- but no love! Apparently Word's HTTP request bypasses
cache. (I verified that the headers returned by the server permitted it to
be cached, again using WireShark; the request is definitely cachable.)

A rational reason for this behavior exceeds the limits of my imagination!
You'd think that if there was even a chance of redundant downloads, they
would at least respect the cache control headers sent by the server, but
nooooo. Cache would've made it barely noticable...

Anyways, the problem isn't a show-stopper, but the needless waste of server
resources (oh yeah, and end-users' time) is always annoying... hoping maybe
someone here has some insight.


TIA,
Mark McGinty

Peter,

I was hoping you would reply! :-) You may recal we discussed related
issues 6-8 months ago? Your input was quite valuable to say the least.


"Peter Jamieson" wrote in message
...
I can't answer your question, but could not get to the point where Word
would open my csv from an http address so could not look at the traffic.

Maybe you could let me know precisely what changes you ended up making in
your xml.

In a nutshell, I set w:mailMerge/w:query-w:val to "SELECT * FROM " & Url,
and w:mailMerge/w:dataSource-w:val to Url, then build the field structure.
To tell you *precisely*, VBS code is below -- note that I edited it to
remove the convoluted bits that aren't especially relevant. Stuff in {} are
tokens to represent the end result of some processing. (Hopefully I didn't
make any syntax errors.)
__________________________________________________

[Apparently VBScript code isn't allowed in posts anymore? This is the third
time I've posted this, except this time without the code. I can email it to
you privately, or I can stick it on my web host somewhere, whatever you
prefer.]
________________________________________________

The blank doc was created by merging with a local CSV -- and possibly
hacked/mangled in some way by hand after, I can't recall. It does open in
Word without modifications, given that same local file exists.

*Trick when analyzing WordXML: comment the second node, so it loads in IE
nicely formatted.

Something I just noticed, when I save the doc to file, Word replaces
w:mailMerge/w:query-w:val with "SELECT * FROM {8-digit hex}.asp" (Not
necessarily relevant, but certainly interesting.) So apparently it creates
something on the client system, however, the data is definitely dynamic
every time the document is opened. (The same SQL is used, but the same
criteria could return a different set of records tomorrow, than it did
yeaterday.)

An aside, last time we corrsponded I had given up on using HTML as the
format, couldn't get it to work, but in re-testing/re-validating my
observations/assumptions, I realized I must've given up while trying to get
the UI to use an HTTP-based source. I should probably experiment more with
it. I saw that HTML the value of w:mailMerge/w:query-w:val is "SELECT *
FROM table". What to infer from that, I'm uncertain.

Another aside, a couple of times I've accidentally stumbled into a condition
where Word prompts me to save the current merge data, and the dialog allows
you to pick the format. It's some combination of editing the data in Word
and then changing something about the source -- I don't know exactly how to
repeat it. In any case, one time it came up I saved the data as XML -- very
ugly stuff, I can't decide if it is more bloated than convoluted or vice
versa, but I can say I have no interest in generating it. :-)

Thanks for the reply!


-Mark

I suspected that it was you who had mentioned the XML approach but
hadn't been back to check until now. Thought the last conversation was
in the autumn - scary! Anyway, my own message has reminded me that the
http:// location is probably a factor.

I'll have a look through your message over the next day or so and see
what else I can find out. By all means despam my email and send me the
vbs - no idea what exactly prevents it from being posted here.

Peter Jamieson

http://tips.pjmsn.me.uk

On 22/02/2010 23:04, Mark McGinty wrote:
Peter,

I was hoping you would reply! :-) You may recal we discussed related
issues 6-8 months ago? Your input was quite valuable to say the least.


"Peter wrote in message
...
I can't answer your question, but could not get to the point where Word
would open my csv from an http address so could not look at the traffic.

Maybe you could let me know precisely what changes you ended up making in
your xml.

In a nutshell, I set w:mailMerge/w:query-w:val to "SELECT * FROM "& Url,
and w:mailMerge/w:dataSource-w:val to Url, then build the field structure.
To tell you *precisely*, VBS code is below -- note that I edited it to
remove the convoluted bits that aren't especially relevant. Stuff in {} are
tokens to represent the end result of some processing. (Hopefully I didn't
make any syntax errors.)
__________________________________________________

[Apparently VBScript code isn't allowed in posts anymore? This is the third
time I've posted this, except this time without the code. I can email it to
you privately, or I can stick it on my web host somewhere, whatever you
prefer.]
________________________________________________

The blank doc was created by merging with a local CSV -- and possibly
hacked/mangled in some way by hand after, I can't recall. It does open in
Word without modifications, given that same local file exists.

*Trick when analyzing WordXML: comment the second node, so it loads in IE
nicely formatted.

Something I just noticed, when I save the doc to file, Word replaces
w:mailMerge/w:query-w:val with "SELECT * FROM {8-digit hex}.asp" (Not
necessarily relevant, but certainly interesting.) So apparently it creates
something on the client system, however, the data is definitely dynamic
every time the document is opened. (The same SQL is used, but the same
criteria could return a different set of records tomorrow, than it did
yeaterday.)

An aside, last time we corrsponded I had given up on using HTML as the
format, couldn't get it to work, but in re-testing/re-validating my
observations/assumptions, I realized I must've given up while trying to get
the UI to use an HTTP-based source. I should probably experiment more with
it. I saw that HTML the value of w:mailMerge/w:query-w:val is "SELECT *
FROM table". What to infer from that, I'm uncertain.

Another aside, a couple of times I've accidentally stumbled into a condition
where Word prompts me to save the current merge data, and the dialog allows
you to pick the format. It's some combination of editing the data in Word
and then changing something about the source -- I don't know exactly how to
repeat it. In any case, one time it came up I saved the data as XML -- very
ugly stuff, I can't decide if it is more bloated than convoluted or vice
versa, but I can say I have no interest in generating it. :-)

Thanks for the reply!


-Mark

Mark McGinty wrote:
Peter,

I was hoping you would reply! :-) You may recal we discussed related
issues 6-8 months ago? Your input was quite valuable to say the least.

Hello again, Mark

Were you not surprised that someone took the trouble to answer you in a
'test' group?

For my part, I've spent more than four years looking for 'unusual'
things on the Internet and investigating what the 'bad guys' do. At
times I have used that particular 'test' group to keep in touch with old
contacts(!)but have been truly amazed at some things people post there.

David H Lipman is something of a Malware guru and frequents many
'security'-type groups but that does not explain why he feels he has a
need to monitor a 'test' group - does it?!!

Maybe someone else can suggest a reason for Mr Lipman's interest in the
'test.here' group.

Regardless, I'm pleased to see that you are obviously up and running
again in *your* group!

Regards,

Dave

The answer was provided via the microsoft.public.word.mailmerge.fields
newsgroup. It was propogated to the text newsgroup because the OP had
cross-posted to that group.

--
Hope this helps,

Doug Robbins - Word MVP

Please reply only to the newsgroups unless you wish to obtain my services on
a paid professional basis.

"~BD~" wrote in message
...
Mark McGinty wrote:
Peter,

I was hoping you would reply! :-) You may recal we discussed related
issues 6-8 months ago? Your input was quite valuable to say the least.

Hello again, Mark

Were you not surprised that someone took the trouble to answer you in a
'test' group?

For my part, I've spent more than four years looking for 'unusual' things
on the Internet and investigating what the 'bad guys' do. At times I have
used that particular 'test' group to keep in touch with old contacts(!)but
have been truly amazed at some things people post there.

David H Lipman is something of a Malware guru and frequents many
'security'-type groups but that does not explain why he feels he has a
need to monitor a 'test' group - does it?!!

Maybe someone else can suggest a reason for Mr Lipman's interest in the
'test.here' group.

Regardless, I'm pleased to see that you are obviously up and running again
in *your* group!

Regards,

Dave

Doug Robbins - Word MVP wrote:
The answer was provided via the microsoft.public.word.mailmerge.fields
newsgroup. It was propogated to the text newsgroup because the OP had
cross-posted to that group.


Hello Doug

I confess to being totally confused by your response!

I first saw a post made by Mark in the *test* newsgroup.

This was his message header:


From: "Mark McGinty"
Subject: so this NG works but the rest of microsoft.public is jacked?
Date: Mon, 22 Feb 2010 08:02:42 -0800
Lines: 2
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.3790.3959
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4325
X-RFC2646: Format=Flowed; Original
Message-ID:
Newsgroups: microsoft.public.test.here
NNTP-Posting-Host: 75-128-3-165.dhcp.snlo.ca.charter.com 75.128.3.165
Path:
border1.nntp.ams.giganews.com!border2.nntp.ams.gig anews.com!nntp.giganews.com!news.unit0.net!news4us .nl!TK2MSFTNGP01.phx.gbl!TK2MSFTNGP02.phx.gbl
Bytes: 706
Xref: number.nntp.dca.giganews.com microsoft.public.test.he7480695


--
Dave - hoping that Doug has mis-read the situation!

If you look at the header of your message, you will see that it is posted to
the following newsgroups:

microsoft.public.word.mailmerge.fields,24hoursuppo rt.helpdesk,microsoft.public.test.here

I replied to it in the first of those newsgroups.

The 24hoursupport.helpdesk newsgroup however cannot be resolved by my news
program so my responses are not being posted to that group.
--
Hope this helps,

Doug Robbins - Word MVP

Please reply only to the newsgroups unless you wish to obtain my services on
a paid professional basis.

"~BD~" wrote in message
...
Doug Robbins - Word MVP wrote:
The answer was provided via the microsoft.public.word.mailmerge.fields
newsgroup. It was propogated to the text newsgroup because the OP had
cross-posted to that group.


Hello Doug

I confess to being totally confused by your response!

I first saw a post made by Mark in the *test* newsgroup.

This was his message header:


From: "Mark McGinty"
Subject: so this NG works but the rest of microsoft.public is jacked?
Date: Mon, 22 Feb 2010 08:02:42 -0800
Lines: 2
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.3790.3959
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4325
X-RFC2646: Format=Flowed; Original
Message-ID:
Newsgroups: microsoft.public.test.here
NNTP-Posting-Host: 75-128-3-165.dhcp.snlo.ca.charter.com 75.128.3.165
Path:
border1.nntp.ams.giganews.com!border2.nntp.ams.gig anews.com!nntp.giganews.com!news.unit0.net!news4us .nl!TK2MSFTNGP01.phx.gbl!TK2MSFTNGP02.phx.gbl
Bytes: 706
Xref: number.nntp.dca.giganews.com microsoft.public.test.he7480695


--
Dave - hoping that Doug has mis-read the situation!

From: "Doug Robbins - Word MVP"

| If you look at the header of your message, you will see that it is posted to
| the following newsgroups:

| microsoft.public.word.mailmerge.fields,24hoursuppo rt.helpdesk,microsoft.public.test.
| here

| I replied to it in the first of those newsgroups.

| The 24hoursupport.helpdesk newsgroup however cannot be resolved by my news
| program so my responses are not being posted to that group.
| --
| Hope this helps,

| Doug Robbins - Word MVP


Actually, since the MS News Server propogates to Usenet so does your message and thus it
does propogate to to the Troll infested group; 24hoursupport.helpdesk

And BoaterDave is quite happy in that group as he is a troll and x-posted this message
there were it did NOT belong. Ask your fellow MS MVP personnel in the private MVP area if
you need confirmation on BoaterDave (aka; ~BD~).

--
Dave
http://www.claymania.com/removal-trojan-adware.html
Multi-AV - http://www.pctipp.ch/downloads/dl/35905.asp

Andy wrote:
Doug Robbins - Word MVP wrote:


If you look at the header of your message, you will see that it is posted to
the following newsgroups:

microsoft.public.word.mailmerge.fields,24hoursuppo rt.helpdesk,microsoft.public.test.here

I replied to it in the first of those newsgroups.

The 24hoursupport.helpdesk newsgroup however cannot be resolved by my news
program so my responses are not being posted to that group.

Yes they are.



You are 100% correct, Andy. Thanks for posting.

Doug Robbins seems to be an "A-OK" MVP but has obviously *no clue* about
what is happening here!

I saw a message in the 'microsoft.public.test.here'group - nowhere else.
I responded to it ONLY in the *test* newsgroup. Here is my header to
substantiate this fact:-


Path:
border1.nntp.ams.giganews.com!feeder2-2.proxad.net!proxad.net!feeder1-2.proxad.net!74.125.46.134.MISMATCH!postnews.googl e.com!news1.google.com!npeer03.iad.highwinds-media.com!news.highwinds-media.com!feed-me.highwinds-media.com!border1.nntp.dca.giganews.com!nntp.gigan ews.com!local2.nntp.dca.giganews.com!nntp.bt.com!n ews.bt.com.POSTED!not-for-mail
NNTP-Posting-Date: Mon, 22 Feb 2010 12:17:03 -0600
Date: Mon, 22 Feb 2010 18:17:04 +0000
From: ~BD~
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US;
rv:1.9.1.8) Gecko/20100205 SeaMonkey/2.0.3
MIME-Version: 1.0
Newsgroups: microsoft.public.test.here
Subject: so this NG works but the rest of microsoft.public is jacked?
References:


In-Reply-To:
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Message-ID:
Lines: 27
X-Usenet-Provider: http://www.giganews.com
X-AuthenticatedUsername: NoAuthUser
X-Trace:
sv3-kyIky1vWwptqNm1lbQdx3tnWqoyAH/ulP8b3Yg8vi08+aKO3HdSzKTGbXzSKEnSraAvbql5xdqcKSjU! QwpxVQ/rhGv6tZa+AlvCeGnPuCNNbxqB2oK5rSAO7IAfYNcBVnjPxETxw QoTi+9nNM6c5/8OTxHN
X-Complaints-To:
X-DMCA-Complaints-To:
X-Abuse-and-DMCA-Info: Please be sure to forward a copy of ALL headers
X-Abuse-and-DMCA-Info: Otherwise we will be unable to process your
complaint properly
X-Postfilter: 1.3.40
Bytes: 2322
Xref: number.nntp.dca.giganews.com microsoft.public.test.he7480698

Mark McGinty wrote:
wrote in message
...

What is your problem, Mark?

Posts to word.mailmerge.fields have not shown up (it's been about 14
hours.)
Also I've noticed that some usually very busy groups have no new
posts in a
relatively long time, e.g, sqlserver.programming, newest post: 2/2082010
12:39 PM.

Maybe it's just my usenet client that's jacked, but I
unsubscribed/re-subscribed to all the groups I read...

Something's wrong somewhere.


-Mark



Try a post here, Mark: 24hoursupport.helpdesk

There are some good guys who might help!

--
Dave