HomeGroupsTalkZeitgeist
  • LibraryThing
  • Book discussions
  • Your LibraryThing
  • Join to start using.

All-Time tag combining stats are wrong on Zeitgeist page

Bug Collectors

Join LibraryThing to post.

1SilentInAWay
Nov 7, 2010, 9:22pm Top

On the zeitgeist Helpers page, the list of All-Time Top 50 tag combiners is missing all combinations prior to the most recent update.

2keristars
Nov 7, 2010, 9:30pm Top

Yeah, it used to just read "non-members", though. Now there are actual users listed.

3SilentInAWay
Nov 7, 2010, 9:38pm Top

It had actual users for years--even when tag combining was temporarily disabled.

The list contained only the "non-members" count for only a few months (I believe this was reported by someone when it first happened, but I can't find the thread now).

4SilentInAWay
Dec 30, 2010, 4:09pm Top

*bump*

It's really depressing that hundreds of hours of effort can be so easily dismissed.

5vpfluke
Dec 31, 2010, 10:22am Top

#4

It seems to show my tagging effort closer to reality than yours, which is truly more extensive than mine. It seems to only show for me the two most recent "pushes" for tags, rather than going back even to maybe March of this year, more or less when the current sweep of tagging started.

6SilentInAWay
Edited: Dec 31, 2010, 3:46pm Top

To those who find the following comments inappropriate within a bug thread, I ask forgiveness. I nevertheless feel the need to argue why earlier tag combining efforts should not be dismissed in the zeitgeist statistics (as for medals -- bah, humbug! -- it's not recognition, but rather appreciation that is lacking).

I was closing in on 15,000 successful tag combinations when the numbers disappeared. It was a lot more work combining tags before the voting system was introduced--for a long while, you had to combine by entering the appropriate "combination" URLs and LT would navigate to a valid page if the tag combination was successful (I wasn't exaggerating above when I said "hundreds of hours"). Before that, there was a tag combining feature in the LT interface, but it was difficult finding combinations (a few years back, however, it was possible to search using an asterisk as a wildcard and receive a list of all tags that matched that template--the search engine has been remodeled at least twice, maybe three times since that period, however).

There was a jump in tag combining when the work page was modified so that you could list all tags (rather than just your own and the fifty or so most popular). A third, even bigger wave came about a year ago when tag combining (which had been shut down complately for a while) was reintroduced using the current voting system. This is, in my opinion, an excellent improvement over the old system, under which the heavier combiners followed a common practice surprisingly similar to that in place today, yet many casual combiners would lump together tags that should not, by consensus, have been combined. There were many conversations--both in Combiners! and (more importantly) in private, with a few active individuals acting as watchdogs (including an extremely vigilant watchcat!!), pointing out inappropriate tag combinations. Back then, etiquette demanded that tags remain separate if anyone had a legitimate argument against combining them. This was important because the overall tag system in LT did not have the flexibility that it has today (when tags were combined, you could not list the books that had been assigned each individual tag--including your own). Fortunately, in those days, you could separate tags almost as quickly as you could combine them.

The present jump in combining activity (the biggest yet) resulted from the perfect storm of a search engine that allows potential tag combinations to be unearthed with unprecedented ease and a tag-combining facility that is easy to understand and easy to operate. Unfortunately, with so many members who are new to combining, this ease has sometime lead to a number of indiscriminate proposals that (according to the previously established common practice) should not be combined. With so many proposals, voting is like--if you forgive me an old vulgarity--"picking the flyshit out of the pepper." Specific cases continue to be discussed, but the sheer mass of proposals (many of which I have been and will continue to be guilty of introducing) makes this difficult. Fortunately, under the current system, a consensus is not needed to oppose a combination, only a substantial minority; the sheer number of proposals, however, are scaring away a good number of responsible voters (I truly hope that they will come back when storm subsides).

At any rate, the zeitgeist totals no longer reflect any of the tag combining efforts prior to the most recent onslaught. Not only are tens of thousand of combinations from the past completely ignored, but also thousands of combinations that were effected after the current voting system was established.

It is difficult staying dedicated to a long-term activity such as combining when the only trace of your efforts can be erased at any time.

7BarkingMatt
Dec 31, 2010, 4:01pm Top

You're absolutely right and I very much respect your effort. Also I find it staggering that I now seem to be one of the the top tag combiners. I'm doing my bit under the new system, but that's simply untrue.

8timspalding
Edited: Jan 3, 2011, 1:35am Top

I've added back the numbers from before the system was changed, and added them to the current numbers.

Those earlier numbers are solid but, frankly, I'm not sure about tracking this going forward. The key problem is that tag combinations are now proposed, and the actual combination is done in a complicated way by code. I think the answer is to take the old numbers and add all successful proposals to them.

Although they were showing up, the actual combinations are no longer as solid as before, because the system rejiggers whole "clusters" of tags to point to a new tag when the lead tag in a cluster changes--a process that both in code and logically renders individual contributions somewhat muddy.

9SilentInAWay
Jan 3, 2011, 5:54am Top

Thanks, Tim...however...

All 52,370 combinations from the past are now being credited to Non-members, rather than being allocated to the specific members who did the work. When the voting system was first introduced, these numbers displayed correctly for six months or so, and then were combined into Non-members (as they are now) for a few months before going away altogether.

8> counting successful proposals, rather than actual combinations

Although different, I think that the number of successful proposals is an appropriate way to track level of effort--provided that by successful you mean passing. Duplicate and circular proposals (situations in which the two methods would result in different counts) are not only relatively rare, but certainly not a serious enough issue to warrant special handling.

Thanks again for giving this your attention, Tim.

10timspalding
Edited: Jan 4, 2011, 12:43am Top

>9

No, they're double counted. I'm combining two data sources--the database as it was when we changed and now. The now database includes all previous combinations, but they are registered to no user. That's why you have 16k or so--the 15k you had before and the almost 2k you've made since.

Does this make sense? Am I wrong?

11SilentInAWay
Jan 4, 2011, 2:17am Top

Makes perfect sense -- I'm just not seeing it.

On the Helpers zeitgeist page, my "All-time" tag combining total is only 2,276 (and my "This week" total is 2,033).

Are we looking at different pages? or perhaps your update has not yet kicked in for regular members?

12SilentInAWay
Jan 12, 2011, 11:13am Top

*bump*

I still see no change in the numbers (other than the Non-Members count now appearing) -- was this fix pushed?

13timspalding
Jan 12, 2011, 8:18pm Top

Ah. See now.

14SilentInAWay
Jan 12, 2011, 8:49pm Top

Yup, I see it now. -- Thanks, Tim!

15_Zoe_
Jan 12, 2011, 8:50pm Top

Close?

16SilentInAWay
Jan 12, 2011, 8:54pm Top

For tracking purpose, wouldn't it be better if Tim marks it as fixed, rather than us marking it as closed?

17_Zoe_
Jan 12, 2011, 9:01pm Top

Yes, but I think he eventually checks all bugs closed by members anyway. So I'm not sure whether it's better to get it off the hot bug list right away or wait until he revisits it. And when I tried earlier this evening to ask him a simple question about how members should deal with bugs, he ended up going through all of them to explain their status ;)

18SilentInAWay
Jan 12, 2011, 9:06pm Top

Personally, I have no problem with this looking like it's still open until LT personnel get around to marking it as fixed. At least they'll eventually get credit for having corrected the problem. Why bother to define different statuses if they're not used...

That said, I'm not invested in the status tracking...mark it as closed, if you wish. I'm just glad that it's fixed.

19_Zoe_
Jan 12, 2011, 9:12pm Top

Well, they would still get credit if they checked the "closed by member" bugs and changed to fixed. But I'm not too worried either way.

20SilentInAWay
Jan 12, 2011, 9:16pm Top

ditto

21timspalding
Jan 13, 2011, 12:26am Top

Update on why this isn't fixed yet (see http://www.librarything.com/topic/107331).

Are you happy with the result SIAW?

22SilentInAWay
Jan 13, 2011, 3:51am Top

yes sir. thank you.

23_Zoe_
Jan 13, 2011, 9:03am Top

Okay, I'm just going to close this one since it seems like it won't go away otherwise.

24SilentInAWay
Sep 29, 2011, 12:36pm Top

In a recent update to the Zeitgeist Helpers page, the all-time tag combiners stats were changed from all-time successful tag combinations to tag proposals from February 2011 on.

Is there a discussion of this change somewhere?

If not, then why (other than for convenience) was it was decided to throw away the results of hundreds of hours spent by members back when tag combining was much more difficult than it is today?

I realize that this feature (the zeitgeist numbers) was broken (again) prior to this change -- but at least, after the previous fix, the numbers reflected the time and effort spent combining tags over a number of years.

With the new method, I could increase my number a thousand or so by quickly searching for a few common names (e.g., "Washington") and then spending an hour or so firing off spurious tag combination proposals for every tag that contains that term. Most of them would get voted down, of course--but that's beside the point.

A few years ago, finding and executing the same number of tag combinations would literally have taken dozens of hours. When tag searching was easy, tag combining was difficult. Then, later, when tag combining became easy, tag searching was hard. Now that both finding potential tag combinations and submitting combination proposals are both relatively easy, work that used to take days can be accomplished in minutes (really!!). This is a good thing. And the top tag combiners, no doubt, still spend hundreds of hours -- they just accomplish a lot more (for better or worse) during the same time!

In addition to this, the "rules" about what tags should be combined were more strict two or three years ago (prior to voting). Once upon a time, if two apparently synonymous tags were used differently by different members, it was common courtesy to leave them uncombined. The rule was "if in doubt, do not combine." Since then, because there have been improvements to the ways in which members can work with their own tags, idiosyncratic usage is no longer a show-stopper. There has also been a major shift in philosophy among tag combiners (perhaps inevitable with the introduction of voting). In the early years of my LT membership, there were discussions of how the emergent categorization that results from member tagging captures unplanned subtleties that top-down ontologies fail to address. Part of the richness was that slight differences in wording could lead to significantly different sets of books. Now, there is more of a tendency to lump similar tags together--as if creating bigger and bigger buckets is always a good thing.

Before I digress further, the point is that more tag combinations are considered acceptable than were a few years back -- another factor in why tag combining used to take a lot more effort then than now.

To count only the (relatively recent) tag proposals on the Zeitgeist helpers page is unfair to those who put in the time for years prior to the adoption of the latest system. Personally, the number of proposals listed for me on that page is still a pretty big number (even though I haven't focused on tag combining for months). There are other members, however, who used to appear prominently on that list who now no longer receive recognition for their past efforts. In short, the current approach dismisses the significant contributions made by members back when tag combining was not only more difficult, but was not considered "cool."

Sad.

25vpfluke
Sep 29, 2011, 1:15pm Top

I agree that there should be more credit given on the Zeitgeist pages to taggers. Part of the more liberal allowance of tagging could come from the fact that there are a lot more tags on a lot more books, so the idiosyncracy of a given tag is not as apparent. Note, that under the current tagging volume (1.5 years), we are almost at a 100,000 that have been "closed".

26brightcopy
Sep 29, 2011, 1:17pm Top

Maybe there could just be some other stat for "Old Style Tag Combination" that is a snapshot of the stats before the new system went into place.

27timspalding
Sep 29, 2011, 3:08pm Top

I'll take a look at what's possible.

28SilentInAWay
Sep 30, 2011, 5:57pm Top

Thanks, Tim.

29lemontwist
May 21, 2012, 11:54am Top

I thought this was fixed already??? Tim???

30SilentInAWay
May 21, 2012, 12:47pm Top

Nope. My tag combining statistics on the Zeitgeist page still do not include combinations that were effected (and counted on that page) prior to the update mentioned in message 24 above. I'm guessing that somewhere between 10,000 and 15,000 of my own completed combinations are no longer being counted...plus thousands of combinations from other long-time tag combiners like Edward and DisassemblyOfReason.

I realize that all contributions to LT are for the greater good and that recognition is secondary (not to mention arbitrary). Nevertheless, it's been very disillusioning to realize that traces of one's older efforts (in my case, hundreds of hours...back when tag combining was somewhat difficult) can just disappear like this.

31eromsted
May 21, 2012, 3:33pm Top

Here's the last Wayback Machine archive of the "Zeitgeist: Helpers" page under the old system. It's not perfect because there's a big gap in the Wayback dates so some work is still missing. But if LT doesn't have it's own records I'd fully support manually adding these old numbers to the current totals.

32SilentInAWay
May 21, 2012, 6:51pm Top

Yeah...if the current Zeitgeist page truly lists proposals from February 2011 onward, then there's a gap of about a year between that January 2010 Wayback Machine capture and the current numbers. Since I was tag-combining pretty heavily back then, that could easily make up the difference between the 7500 shown in that capture and the larger number floating around in my memory. Of course, I am getting older and my memory isn't exactly what it used to be...

Yes, it would be really nice if LT's own archives could be used to derive accurate (or even semi-accurate) numbers from the past. If not, I'd be happy to see the numbers reflect even partial credit for all that older work.

I realize, of course, that we're talking apples and oranges here -- the current totals reflect combination proposals, whereas the old numbers accounted for completed tag combinations. It would still be extremely encouraging if the older tag combining work were to be somehow accounted for in the Zeitgeist numbers ... if Tim could come up with a fair way of including all of the fruit (the newer apples and the older oranges) in a single basket.

This is not a question of recognition or credit. As I've mentioned elsewhere, there are several reasons that I'd actually argue against a Helper Badge being awarded for either tag proposals or voting (as is done for other combination work). There is, nevertheless, a sense of personal satisfaction at seeing one's numbers rise over the years -- if nothing else, it allows one to entertain the illusion that the hours spent doing this dorky combination stuff actually mean something. I'd find it hard to believe that I'm the only member for whom the Zeitgeist numbers provide a source of positive reinforcement -- even though I appear to stand alone in expressing my discouragement at a good chunk of these numbers having gone away.

Extensive combining on LT is, after all, hardly something one can speak about with pride to one's spouse or friends (without unwanted repercussions, that is)!

33michtelassn
Dec 26, 2012, 10:39am Top

I'm not sure this is the right place for this: In Zeitgeist (http://www.librarything.com/zeitgeist) it shows top 50 taggers with a range of 137,363 to 7,587 tags each. In my home page (http://www.librarything.com/home/michtelassn) I show that I have 16,773 tags. But I do not appear in the Zeitgeist tag list of top 50 taggers.

34vpfluke
Dec 27, 2012, 12:14pm Top

#33

I am wondering if the Zeitgeist page shows only the number of unique tags, and the Home page shows the full amount of tags you've used. My numbers are around 9,200 on the Zeitgeist page, and 39,000 on my Home page.

35jjmcgaffey
Dec 29, 2012, 2:11am Top

If so, you should be able to see the unique tags as well on your Home page - it's directly below the Tags number. Unfortunately no one can see another user's Home page, so each person will have to check for him/herself. The tags info on the Stats/Memes page has total tags, but not unique tags.

I checked the top person in the Top 50 Taggers - on his Stats/Memes page, he's got 257000+ tags instead of the 137000+ listed on the Zeitgeist. That could be delay, or it could be unique tags (he's got a _lot_ of them).

36malinablue
Edited: Dec 30, 2012, 11:28am Top

I have about 54,000 tags and have been on the top 50 taggers list ever since I started tagging. But today I see I am not on the list at all anymore, despite having more tags than most of the list. I think it would be ridiculous to change the metric to "unique" tags. Does anyone know if this is a bug or a change?

cindy

37vpfluke
Dec 30, 2012, 5:25pm Top

I relooked and I have 3 tagging numbers.

On Zeitgeist, under top 50 Taggers, I have 9,201

On my home page, I have done 39,636 total tags, with 12,805 distinct tags.

So, maybe 9,201 is the number of tags I've done in the last 12 months.

38BarkingMatt
Edited: Dec 30, 2012, 5:35pm Top

I don't show under the top 50 taggers, but: 18,368 tags and 1,688 distinct tags apparently (I didn't count them). So, I guess it's not raw numbers but number of distinct tags - which somehow makes sense to me.

Group: Bug Collectors

703 members

66,458 messages

Bug Tracking

This topic is part of LibraryThing's in-talk bug tracking.

Join or watch Bug Collectors to get "Bug Tracking" under "The World" in Talk all the time.

Bug (edit)

ID: 101996

Category: Zeitgeist

Assigned to all

Reported by SilentInAWay

Status: Needs discussion

Sep 29, 2011, 12:36pm

2 years since last change

By SilentInAWay

Status log

Reported. SilentInAWay (Nov 7, 2010, 9:22pm)

Closed by member. _Zoe_ (Jan 13, 2011, 9:03am)

Needs discussion. SilentInAWay (Sep 29, 2011, 12:36pm)

About

This topic is not marked as primarily about any work, author or other topic.

Touchstones

No touchstones

Help/FAQs | About | Privacy/Terms | Blog | Contact | APIs | WikiThing | Common Knowledge | Legacy Libraries | Early Reviewers | 86,415,446 books! | Top bar: Always visible
http://www.librarything.com/topic/101996
www.librarything.com
crawl=CC-MAIN-2013-48
50243
application/http; msgtype=response
sha1:JXA3IAAHRLL5VH72GVQ736XFOKYHOKKC
<urn:uuid:cfa6e658-4fe3-4c7a-a9eb-819bc7b3c17e>
2013-12-09T14:58:41Z
74.201.105.31
sha1:WT32SRRHK4XBYZRIPQC4XOQZL5VC6SEB
<urn:uuid:c6dcaf23-8a69-4816-a3ac-47a6b0451c58>
http://www.librarything.com/topic/101996
length
response
<urn:uuid:47cd6f5b-cdab-460f-99f9-47aa3c0fa838>
robots: classic
hostname: ip-10-33-133-15.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2013-48
operator: CommonCrawl Admin
description: Wide crawl of the web with URLs provided by Blekko for Winter 2013
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf