SES NYC 2006, Day 3

Today I’m trying to catch up on email. Random thoughts:

– Bill Slawski, where are you? Introduce yourself to me! Nick Wilson? Andy Hagans? I’m trying to talk to different people this time; where R U?
– I sat in the “Auditing paid clicks” session the other day. It was in the same room as SES NYC 2005 last year. Last year the room seemed more crowded; I thought there were fewer people in the room this year.
– Aaron D’Souza tackled the Search Engine Q&A on Links session, and did great. It was really nice to sit in the audience and listen, except for:
– I just paid $9.95 for a day of Wifi up in my hotel room. Now you want me to pay $2.95 for 15 minutes, and $0.25 for each additional minute? You seriously want me to pay $15/hour for Wifi after I’ve already paid for it?! $%#@$% !! I hate it when there isn’t free wifi blanketing a conference. Where do I get this EVDO equipment that’s so popular with the kids these days? I can see the dialogue now:

“Why, back in my day, I had to pay $8.70 just to surf for 40 minutes at a conference!”
“Whatever, Grandpa. I bet back then, you still had to walk five miles uphill in the snow just to get to an RSS feed.”

Okay, it’s almost 3 a.m., which means that it’s closing time on preparing for “Meet the Crawlers” tomorrow morning. The Sitemaps team just did another new release (which follows the addition of robots.txt checking tools a few weeks ago), so maybe I’ll try to show some of those features using my site as a guinea pig.

34 Responses to SES NYC 2006, Day 3 (Leave a comment)

  1. I really need to get my ass to that conferance next year. Seems like there is lots of good things to learn there.

  2. I think “Meet The Crawlers” will have somthing to filter BLOG/Comment/Guestbook/Subdomain spamming. I have lost hope for optimization for my industry (you would not like the name). Although it hurts people like us, who are working for clients in genuine way.

  3. Matt

    People are seeing a mad de-indexing of sites on Big Daddy – with supplimentals being added back into the index while freshly crawled pages are being dropped or not making it to the Big Daddy index.

    Is this all part of the process – should we be worrying at this stage or wait until things have rolled out and had time to settle on Big Daddy.



  4. Love the new sitemaps stats. The sitemaps team are obviously working really hard on this. It’s great to see the postions of your top search queries. I wonder what will be next?

  5. $15 an hour for WiFi? Sounds like you’ve come to Europe? ๐Ÿ˜‰

  6. What Stephen said

  7. Hi Matt,

    It’s been an active and busy conference. I’d be pleased by the chance to meet up with you. I’m leaving sometime this afternoon, but I’ll keep my eyes open, and see if I can catch up with you before then.

  8. You need to join the Hilton Honors club – every time I log on in my room it tells me I get the first day free. Apparently, each day has been my first day this week. ๐Ÿ˜‰

    Course, I’m paying $15.00 an hour to type this atm. LOL.

  9. EVDO from Verizon Wireless, Matt, and you’ll never go back. Hey, then you can call me free, ‘cuz I’m IN!

  10. Aaron did a great job yesterday and it is nice to see him stepping up.

  11. I’d (heart) an answer to Stephen’s question too. Thanks Matt!

  12. Matt, it is perhaps another department of Google, but why not implement mobile WIFI installations on behalf of Google on these occasions.

  13. Matt, please don’t type in AOLesque ‘Net abbreviations anymore. It’s beneath you. Come on man, you know better. ๐Ÿ™‚

  14. Yeah that Hilton is really pretty rough, I checked out after day one, and moved over to the Shoreham. Its right around the corner at 33 west 55th, free wifi everywhere, the food is much better and the same price. They even have a free PC with printer in the lobby (P.S. Stay clear of the Hilton BLT YUK!!) And the Shorehams hip hotel lobby bar looks 10 times cooler, and seems to have attractive gals hanging out in it all the time.
    Oh and by the way the best French Food in the entire state of New York is also on that street at La Cote Basque 60 west 55th, I strongly recommend the Lobster Bisque, and the Sea Bass and they both go great with the Ramey Chardonnay at only $140 per bottle.

    P.S. While the assortment of bottled drinks and the raw convenience at the 24 hour deli on the corner of 56th is fantastic (they sell beer 24 hours a day) do not eat from the buffet no matter how drunk or hungry you are or how late it is the food is made of nothing but bad bad mojo.

  15. Agreed. One of the speakers yesterday actually had “U R” on one of her PowerPoint slides. Twice. I was forced to walk out.

    By the way, and I know this is completely off-topic, but I was wondering something about the emailing webmasters program. Is this something that Google is really doing right now, or is it just in the works? Also, are these emails just for sites that have been removed from the index or also sites that have been penalized in other ways?

  16. I’m sitting next to Bill now – Trianon Ballroom, SEO Overkill session. Andy was around here too.

  17. Well Matt,
    The wifi thing is realy something people should think of when organising Seminars. On our topic list is one key factor since the first seminar. WIFI has to be there and has to be free… But well, sometimes this is realy more then hard to get. Not only in New York, also in “snowy, freezing” Switzerland.

  18. Love the new sitemaps features. Wish that BigDaddy would pick up more than 1 of the pages that the crawler team has successfully spidered.

    Any feedback on this Matt? It seems like these indexing problems are more widespread the the usual sites.

  19. Quote:
    โ€œWhy, back in my day, I had to pay $8.70 just to surf for 40 minutes at a conference!โ€
    โ€œWhatever, Grandpa. I bet back then, you still had to walk five miles uphill in the snow just to get to an RSS feed.โ€


    Matt, I did a report in school that contained the EVDO network. Did you know they are coming out with a Revision B that will allow up to 73.5 mbps throughput ๐Ÿ™‚ They say it will be backwards compatible with the current hardware.

  20. I really love the new sitemap stats… good work by the sitemap team.

  21. EVDO is the way to go for so many reasons. I use it, but then you know that from the reverse IP lookup. I’m connected immediately when my Tablet PC comes out of hibernation, no matter where I am.

    The Hilton actually had a dead spot way back in the main Hotel Bar (figures) and I kept getting disconnected, however I routinely get 1MBs down and 300K up.

    It is great in big cities like NY, Seattle, San Jose, and Boston. I just tested it at 1500kbs down and 100kbps up here in Boston. It has a lot of latency though, and you can often (but not always) tell the difference between wifi or cat5 connection. It is a worthwhile alternative to dial up in rural areas but not to broadband – I get only 30k when I’m up skiing in Killington VT.

  22. Matt,

    A lot of wikipedia links within the Google search are not correct.

    For example: for [Niels Bohr biography] a link at the top to wikipedia is placed. In the green uri the link is correct, however, the actual link to the webpage is not.

    The link brings you to instead of Bohr

    (The ” ” is replaced with “+” in the link.)

    I have noticed this is what the links look like for all wikipedia uri’s which have a space in them.

  23. Omming_in_the_dark

    I am new to sitemaps. I thought I should check it out after an essential page from my site disappeared yesterday.

    Sitemaps showed my site could not be verified because I was using an .htaccess file to redirect 404’s (errordocument 404/). On closer inspection, I noticed that the custom 404 was the page that disappeared. I went back to the index and realized that googlebot had indexed an error page in its place (inexistent), 404’d by the htaccess. For a brief period, there were 2 different urls for the same file in the index and as it happens, the original page was knocked out of the index due to duplication.

    I am grateful there is such a thing as ‘sitemaps’ which help me find about the problem. But neither sitemaps nor any other tool could show me how to solve the problem. Not even if it *was* solvable, or a timeframe for the page to come back.

    My immediate response was to kill the line errordocument 404/ in my htaccess file and let the world -including googlebot- get to any and all 404 pages ‘not found’. I have also 301’d the error page indexed by google to the original file, which since last night has been de-indexed.

    Matt, could you be so kind to provide any guidance? I did the reinclusion request for the missing page but got no answer yet… Truly appreciate your words.

  24. I, too, am inquiring about this massive de-indexing of sites. Had #1 positions for several years and a very popular portal of 10 years suddenly lose most of it’s pages. Only page showing up is the home page when doing the “” query. All other results are “supplemental” pages that have a cache from last year.

    I hope I shouldn’t worry and this is just normal for Big Daddy!


  25. Whats up!!!

    One day a 7.5k page site has 800 pages cached, the next day reporting 8300 on a live datacenter and they are all really old url’s – all supplimentals. However, other datacenters have current url’s and proper caches.

    Search phrases that have been top now don’t exist since the pages are not cached. Is this Bigdaddy muck ups that we have to ride out?

    Not very nice when I have to survive on msn and yahoo number one serps. They are seeming to be far more dependable but have low traffic.

  26. Yeah, Matt – why don’t you make Google more dependable? Geeze!

    BTW – Heard from Bill that you guys ran into each other; he’s a swell chap, eh?

  27. Sorry I missed you; I was on my way out of the city before I heard I was on the Mr.-Cutts-wants-to-meet-me list…

    I’d like to get around and do a bit more schmoozing but I have to man the booth when at the shows. Anyway, I’ll try to track you down in Boston ๐Ÿ™‚

  28. >>>>Yeah, Matt – why donโ€™t you make Google more dependable? Geeze!

    So others suffering is happiness for you. The new big daddy pages dropout affected million of sites. Almost 50% of sites in different niche.

    Most of the sites affected just have their homepage listed all legitimate pages dropped out of google index.

    check for the keyword “search engine optimization”

    You can see 40 out of 100 sites affected with this google bigdaddy bug.

    Including sites like , , lost all their pages except their homepage. All other age old pages listed are only supplementary. Is it wrong to ask matt regarding this. You need to stop your bragging dude.

  29. George Chaney

    Hey Matt,

    How about addressing googles policy on the use of the rel=”nofollow” in non blog sites. Because of the large spam issue with domains expiring etc, I will add the rel=”nofollow” to sites I like, but don’t trust long term. What is googles policy on the use in links to other sites.

    Look forward to hearing your words on this matter in the various SEO forums I hang out in.

  30. Thank you for the heads up about the new sitemaps feature Matt. People always say how important it is to be on the first few pages of the search results – this new feature only emphasizes the fact. 16 out of our Top 20 search query clicks were in the top 6 results (not the top 6 pages!). So most people should even forget about being satisfied with results on the 2nd page! Go for the Top 5!

    Anyway, off to read the Google Webmaster Guidelines again and do more work on the site!

  31. Mr C. While you been swanning of to the SES a lot of sites are going supplemental:


    Any word on that if you are back at work today

  32. The new rankings in Sitemaps are very good, but I just discovered a problem with the Sitemaps system.

    I registered a sitemap a while ago, and because the system probes for a 404 before it will verify ownership, I used the .htaccess file to send a 404 when the probe filename was requested. Some servers don’t send 404s, and webmasters don’t have control of the servers.

    It was fine for a while, but then the probe filename was changed, and the sitemap became unverified. What’s the point of that? Why make me go through the process of downloading a huge logfile to get the new filename and set up the .htaccess file again? Presumably, I’ll have to do it every time the filename is changed. Most webmasters won’t even have the knowledge to fiddle their way around the 404 probe, and if the server doesn’t send a 404, they are stuffed..

    Why doesn’t the Sitemaps team do what people have suggested ever since the probe was inroduced – i.e. use a verification that all webmasters have easy control over, such as some unique text in the probe file, or a unique line in the robots.txt file?

  33. The WiFi thing is really a pet peeve of mine too. Think about it. Must be one of the most profitable parts of a hotel bill. $10/day? Well how many people do much more than sleep in their room? So you do some work at night for two hours. As soon as you even so much as take your laptop down to the lobby, you’re no longer “in your room,” so you don’t have validated access to the connection. What a rip!

    Regarding the auditing click fraud session, well maybe I tripped over the wrong group of people, but I was going by in the hall and there were 30 people outside the room watching on monitors. Maybe this means people need to be a bit more aggressive about grabbing empty seats rather than trying to cluster around in the back.

  34. EVDO is the only way to go if you want over-the-air broadband. It is capable of DSL or Cable modem speeds. EDGE is more like 56k, but is more widely available. EVDO works great in up to 30 major US cities, elsewhere is falls down to 1XRTT (and very slow). EVDO is Verizon, EDGE is Cingular and T-Mobile.

    The problem I have, is EVDO phones are not GSM and won’t work outside the US. EDGE phones are not as cool as, say, the Treo series. So, what to do? Danny bought a Treo EVDO phone while at the conference on a shopping trip. I broke down and did the same. The thing is, the aircard requires its own service agreement! Verizon really ought to bundle and encourage more purchases that way.