  1. Hi there, Guest

    Only registered users can really experience what DLP has to offer. Many forums are only accessible if you have an account. Why don't you register?
    Dismiss Notice
  2. DLP Writing Competition
    Topic - Master and Apprentice (or Mentor and Protege!)

    Click here for more info!
    Don't miss the due date this time Guest - it's March 13th!
    Dismiss Notice
  3. Introducing for your Perusing Pleasure

    New Thread Thursday
    +
    Shit Post Sunday

    READ ME
    Dismiss Notice

Scryer - DLP's FF Search Engine

Discussion in 'General Discussion' started by Lord Ravenclaw, Mar 31, 2016.

  1. SilverOtter

    SilverOtter Seventh Year

    Joined:
    Feb 20, 2011
    Messages:
    262
    Location:
    USA
    It looks like the search is currently down.
     
  2. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,359
    Location:
    Denver, CO
  3. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,359
    Location:
    Denver, CO
    New feature (one of many to come): open tracking. When logged in, Scryer will log the stories you open for the purpose of telling you which stories you've already opened. Coming soon, a way to clear this data or specific entries you're embarrassed about.

    Screen Shot 2017-10-03 at 21.20.45.png

    Additionally, the site's indexing backend has been significantly upgraded. Along with improved indexing quality, this has lead to significant site speedups (searches are ~53% faster). More to come there as well.

    Next up:
    1) Opt-out/clearing data on story opens.
    2) Search by ship.
    3) Better ship detection, even when the author has not added it to the FFN metadata.
     
  4. oakgem217

    oakgem217 First Year

    Joined:
    Nov 30, 2014
    Messages:
    39
    I tried doing a basic search, but when I tried including the summary keywords (-slash, bashing, etc.), I got a 500 error
     
  5. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,359
    Location:
    Denver, CO
    Can you either link the search or the request ID in the error so I can check the logs?
     
  6. oakgem217

    oakgem217 First Year

    Joined:
    Nov 30, 2014
    Messages:
    39
    RequestId: e7604f7a-29a2-46e1-80a2-7354b7b0b0ba
     
  7. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,359
    Location:
    Denver, CO
    '!' is a special token in Lucene Query Syntax, so '-!' is an invalid query, it'd have to be written as '-\!' to escape the special token. In any case, full-text search doesn't retain special characters like '!', so it's not usable as a part of the query.

    I've made this more clear by capturing the ElasticSearch parse exception and returning a more helpful (still cryptic, but more helpful) error message:

    Screen Shot 2017-10-08 at 13.14.04.png
     
    Last edited: Oct 8, 2017
  8. oakgem217

    oakgem217 First Year

    Joined:
    Nov 30, 2014
    Messages:
    39
    Alright, sounds good :)
     
  9. John1057

    John1057 Squib

    Joined:
    Sep 12, 2013
    Messages:
    5
    Don't know if this has been mentioned elsewhere, but i can't erase fandoms selected. Which basically means i'm stuck searching harry potter every time no matter what. This appears to only affect the very first category of fandoms as every other area i can select a crossover fandom or character or genre and then press backspace or click the x to get rid of it. However harry in the fandoms box can not be deleted.

    Minor work around was that i had some saved searches that fill that box without harry potter, but coming in not signed in, i can't delete the selection.

    This occurs on chrome, internet explorer, and android chrome beta that i've confirmed.
     
  10. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,359
    Location:
    Denver, CO
    Add a second fandom, then you can remove HP. It's unclear to me why the behavior changed, I haven't messed with the frontend in a while.
     
  11. John1057

    John1057 Squib

    Joined:
    Sep 12, 2013
    Messages:
    5
    On Further testing, it may actually be clearing the fandom but not updating what is displayed as evidence here when i can have harry potter 4 separate times

    Hitting backspace does not delete any of those. Hitting the x clears out some but not all. It's very weird.
     

    Attached Files:

  12. zekses

    zekses First Year

    Joined:
    Dec 3, 2015
    Messages:
    33
    High Score:
    0
    Scryer slash tagger probably needs manual exclusions, because it tags fics like "Lily and the Art of Being Sisyphus" as slash. It's perfectly reasonable given how it works, but completely incorrect nevertheless.
     
  13. draykfyre

    draykfyre First Year DLP Supporter

    Joined:
    Sep 13, 2011
    Messages:
    46
    Could you add The Hobbit to the index? Thanks!
     
  14. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,359
    Location:
    Denver, CO
    Should be indexed in a few hours.
     
  15. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,359
    Location:
    Denver, CO
    FWIW, this issue is resolved now.
     
  16. AceOfSpades

    AceOfSpades Slug Club Member DLP Supporter

    Joined:
    Oct 21, 2007
    Messages:
    186
    Location:
    Hot and humid
    Does Scryer have the ability to return users who favorite'd a particular author or story? I feel like I remember that being a feature early on, though it may have been one of Scryer's predecessors.
     
  17. Zombie

    Zombie Black Philip Moderator DLP Supporter

    Joined:
    Apr 28, 2007
    Messages:
    6,015
    There used to be a favorites search
     
  18. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,359
    Location:
    Denver, CO
    Not in Scryer. It'd be something I'd consider if there was enough demand; currently I don't index users.
     
  19. AceOfSpades

    AceOfSpades Slug Club Member DLP Supporter

    Joined:
    Oct 21, 2007
    Messages:
    186
    Location:
    Hot and humid
    It may be useful in a recommendation engine if that's in the cards. It's another dimension of data for measuring story similarity and potential 'likability'.
     
  20. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,359
    Location:
    Denver, CO
    Agreed -- recommendation engine would probably be the only reason I'd index that information. It might be something I do this year, but there are other quality-of-life improvements I'd like to make to Scryer first.
     
Loading...
https://forums.darklordpotter.net/threads/scryer-dlps-ff-search-engine.32979/page-6
forums.darklordpotter.net
crawl=CC-MAIN-2021-04
118620
application/http; msgtype=response
sha1:6YGNB3OHAR3HZMTCQZJSHNLHDCNBDX7Z
<urn:uuid:2561bda7-e65c-40b3-8590-8b5396bc6283>
2021-01-19T21:04:12Z
96.126.120.137
text/html
sha1:NBXP2UBENM4CEGDGFVZY6W5MREQQ67DT
<urn:uuid:d14dfcfa-01ef-47ac-b737-f7aa2b84af32>
https://forums.darklordpotter.net/threads/scryer-dlps-ff-search-engine.32979/page-6
response
<urn:uuid:523dc764-e2f1-4899-861b-ed62490fe35a>
isPartOf: CC-MAIN-2021-04
publisher: Common Crawl
description: Wide crawl of the web for January 2021
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-59.ec2.internal
software: Apache Nutch 1.17 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.2-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/