org.apache.solr.common.SolrException: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840

Apache's JIRA Issue Tracker | Nikolay | 3 years ago
  1. 0

    [SOLR-5426] org.apache.solr.common.SolrException; org.apache.solr.common.SolrException: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840 - ASF JIRA

    apache.org | 11 months ago
    org.apache.solr.common.SolrException: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840
  2. 0

    Highlighter does not work correctly on test-data. I added index- and config- files (see attached highlighter.zip) for reproducing this issue. Everything works fine if I search without highlighting: http://localhost:8983/solr/global/select?q=aa&wt=json&indent=true But if search with highlighting: http://localhost:8983/solr/global/select?q=aa&wt=json&indent=true&hl=true&hl.fl=*_stx&hl.simple.pre=<em>&hl.simple.post=<%2Fem> I'm get the error: ERROR - 2013-11-07 10:17:15.797; org.apache.solr.common.SolrException; null:org.apache.solr.common.SolrException: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840 at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:542) at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:414) at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:139) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:368) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Unknown Source) Caused by: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840 at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:225) at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:527) ... 33 more

    Apache's JIRA Issue Tracker | 3 years ago | Nikolay
    org.apache.solr.common.SolrException: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840
  3. 0

    Highlighter does not work correctly on test-data. I added index- and config- files (see attached highlighter.zip) for reproducing this issue. Everything works fine if I search without highlighting: http://localhost:8983/solr/global/select?q=aa&wt=json&indent=true But if search with highlighting: http://localhost:8983/solr/global/select?q=aa&wt=json&indent=true&hl=true&hl.fl=*_stx&hl.simple.pre=<em>&hl.simple.post=<%2Fem> I'm get the error: ERROR - 2013-11-07 10:17:15.797; org.apache.solr.common.SolrException; null:org.apache.solr.common.SolrException: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840 at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:542) at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:414) at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:139) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:368) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Unknown Source) Caused by: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840 at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:225) at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:527) ... 33 more

    Apache's JIRA Issue Tracker | 3 years ago | Nikolay
    org.apache.solr.common.SolrException: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840
  4. Speed up your debug routine!

    Automated exception search integrated into your IDE

  5. 0

    偏移量错误

    GitHub | 4 months ago | drmam
    org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token , exceeds length of provided text sized 98
  6. 0

    InvalidTokenOffsetsException in conjunction with highlighting and ICU folding and edgeNgrams

    lucene-solr-user | 5 years ago | Max
    org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token mosfellsbaer exceeds length of provided text sized 11

    Not finding the right solution?
    Take a tour to get the most out of Samebug.

    Tired of useless tips?

    Automated exception search integrated into your IDE

    Root Cause Analysis

    1. org.apache.lucene.search.highlight.InvalidTokenOffsetsException

      Token 0 exceeds length of provided text sized 840

      at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments()
    2. Lucene Highlighter
      Highlighter.getBestTextFragments
      1. org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:225)
      1 frame
    3. Apache Solr Core
      SolrDispatchFilter.doFilter
      1. org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:527)
      2. org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:414)
      3. org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:139)
      4. org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
      5. org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
      6. org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
      7. org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
      8. org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
      9. org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
      9 frames
    4. Jetty
      BlockingHttpConnection.handle
      1. org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
      2. org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
      3. org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
      4. org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
      5. org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
      6. org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
      7. org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
      8. org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
      9. org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
      10. org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
      11. org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
      12. org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
      13. org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
      14. org.eclipse.jetty.server.Server.handle(Server.java:368)
      15. org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
      16. org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
      17. org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
      18. org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
      19. org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
      20. org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
      21. org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
      21 frames
    5. GWT dev
      SocketConnector$ConnectorEndPoint.run
      1. org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
      1 frame
    6. Jetty
      QueuedThreadPool$3.run
      1. org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
      2. org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
      2 frames
    7. Java RT
      Thread.run
      1. java.lang.Thread.run(Unknown Source)
      1 frame