Asciidoctor support

[nik9000]

I'm using this to track things left to do for the initial Asciidoctor support:

• Build docs with Asciidoctor when running --all
  • Installation and Upgrade Guide (Switches Install Upgrade Guide to asciidoctor builds #629)
    • The overview page has a different name. Oddly, it looks like asciidoctor is doing the right thing here. I think we can correct this when when we cut over.
  • Getting Started (Switches Getting Started Guide to asciidoctor builds #645)
    • Fix crashing preprocessor when it hits a skipped line Asciidoctor: Handle skipped lines #643
    • ID generation doesn't quite line up (see above)
  • Glossary
  • Stack Overview (Switches Stack Overview to asciidoctor builds #649)
    • Prevent snippets inside definition lists from blowing up (Asciidoctor: Handle snippets in definition lists #680)
  • Deploying with Azure Marketplace and Resource Manager (ARM) template
  • Elasticsearch reference (Switches Elasticsearch Ref doc build to asciidoctor #827)
    • Prepare the book ([DOCS] Fix Asciidoctor errors to prepare for migration elasticsearch#41128)
  • Elasticsearch Resiliency Status (Switch resiliency status to asciidoctor #666)
  • Painless Scripting Language (Switch Painless reference to Asciidoctor #783)
    • Drop inline callouts (Docs: Drop inline callouts elasticsearch#38335)
  • Elasticsearch Plugins and Integrations (Switch ES plugins docs to Asciidoctor #796)
  • Elasticsearch Clients: Java REST Client (Switch Java REST client to Asciidoctor #780)
    • Stop include-tagged from striping out spaces (Asciidoctor: Fix Elastic's include-tagged with empty lines #693)
  • Elasticsearch Clients: Java API (Switch the Transport Client docs to Asciidoctor #778)
  • Elasticsearch Clients: JavaScript API (Updated configuration for js client #701)
  • Elasticsearch Clients: Groovy API (Switch the groovy API to Asciidoctor #696)
  • Elasticsearch Clients: .NET API (Switch .net client to Asciidoctor #812)
  • Elasticsearch Clients: Perl API (Switch the perl client docs to asciidoctor #685)
  • Elasticsearch Clients: PHP API (Switch the PHP client to Asciidoctor #690)
  • Elasticsearch Clients: Python API (Switch the ES Python Client to Asciidoctor #702)
  • Elasticsearch Clients: Ruby API (Switch ruby client docs to asciidoctor #691)
  • Elasticsearch Clients: Community Contributed Clients (Switch the community clients book to Asciidoctor #692)
  • Elasticsearch for Apache Hadoop and Spark (Switch es-hadoop docs to asciidoctor #785)
    • Drop inline callouts (Docs: Drop inline callouts elasticsearch-hadoop#1270)
    • Handle index.adoc (Asciidoctor: Support adoc files #709)
  • Curator Index Management (Switch curator docs to Asciidoctor #835)
  • Cloud: Elasticsearch Service - Hosted Elasticsearch and Kibana
    • Drop inline callouts
  • Elasticsearch Add-On for Heroku - Hosted Elasticsearch and Kibana for Heroku Users
    • Drop inline callouts
  • Cloud Enterprise - Elastic Cloud on your Infrastructure
    • Drop inline callouts
  • Kibana Reference (Switches Kibana doc build to asciidoctor #756)
  • Uptime Monitoring Guide (Switch uptime docs to Asciidoctor #1322)
  • Logstash: Reference (Switch logstash to asciidoctor #1005)
  • Logstash Versioned Plugin Reference (Switch Logstash VPR to Asciidoctor #822)
    • Clean up markdown leaks
  • Beats Platform Reference
  • Beats Developer Guide
  • Packetbeat Reference
  • Filebeat Reference
  • Winlogbeat Reference
  • Metricbeat Reference
  • Heartbeat Reference
  • Auditbeat Reference
  • Functionbeat Reference
  • Journalbeat Reference
  • Legacy Topbeat Reference
  • Site Search Reference (Move the swiftype placeholder to asciidoctor #1281)
  • App Search Reference (Move the swiftype placeholder to asciidoctor #1281)
  • Infrastructure Monitoring Guide (Switch the Infra Monitoring Guide to asciidoctor #698)
  • APM Overview (Switch APM Overview to Asciidoctor #733)
  • APM Server Reference (Switch APM Server to Asciidoctor #1083)
    • Figure out what to do about complex conditionals
  • APM Node.js Agent (Switch APM Agent for NodeJS to Asciidoctor #724)
  • APM Python Agent (Switch Python APM Agent docs to Asciidoctor #723)
  • APM Ruby Agent (Switch APM Agent for Ruby to Asciidoctor #715)
  • APM Real User Monitoring JavaScript Agent (Switch APM Agent for RUM JavaScript #714)
  • APM Go Agent (Switch APM Agent for Go to Asciidoctor #710)
  • APM Java Agent (Switch the APM Agent for Java to Asciidoctor #707)
  • APM .NET Agent (Switch the APM Agent for .NET to Asciidoctor #705)
  • Elastic Common Schema (Switch the ECS docs to Asciidoctor #813)
  • cn《Elasticsearch 权威指南》中文版 (Switch the Chinese Definitive Guide to Asciidoctor #1317)
  • cn PHP API (Switch CN PHP API to asciidoctor #1082)
  • cn Kibana 用户手册 (Switch Kibana's CN book to asciidoctor #1310)
  • jp Elasticsearchリファレンス (Switch the Japanese ES reference to Asciidoctor #1320)
  • jp Logstashリファレンス (Switch Japanese Logstash reference to Asciidoctor #1319)
  • jp Kibanaユーザーガイド (Switch Japanese Kibana docs to Asciidoctor #1315)
  • jp X-Packリファレンス (Switch Japanese XPack docs to Asciidoctor #1199)
  • kr Elasticsearch 참조 (Switch the Korean ES reference to Asciidoctor #1321)
  • kr Logstash 참조 (Switch Korea Logstash reference to Asciidoctor #1318)
  • kr Kibana 사용자 가이드 (Switch Korean translation of kibana to Asciidoctor #1314)
  • kr X-Pack 참조 (Switch kr X-Pack docs to Asciidoctor #1311)
  • X-Pack Reference for 6.0-6.2 and 5.x (Switches X-Pack Reference to Asciidoctor builds #653)
  • Elasticsearch - The Definitive Guide (Convert the definitive guide to asciidoctor #1309)
  • Sense Editor for 4.x (Switch legacy sense documentation to Asciidoctor #746)
  • Marvel Reference for 2.x and 1.x (Switch legacy marvel docs to asciidoctor #842)
  • Shield Reference for 2.x and 1.x (Switch legacy shield docs to asciidoctor #841)
  • Watcher Reference for 2.x and 1.x (Switch legacy watcher documentation to Asciidoctor #821)
  • Reporting Reference for 2.x (Switch legacy reporting docs to Asciidoctor #816)
  • Graph Reference for 2.x (Switch legacy graph docs to Asciidoctor #814)

Archived bullets:

• Fix rendering issues
  • Table of contents doesn't explode (Asciidoctor: Fix raw xslt opts #527)
  • Every page starts with a section containing just a double quote (Asciidoctor: Fix raw xslt opts #527)
  • "Edit me" links didn't appear (Asciidoctor: Add "Edit Me" Links #526)
  • Ending a page with a callout list seems to eat the next line of the next page. (Asciidoctor: Work around "cramped" include statements #529)
  • Some snippets aren't escaped properly generating invalid xml
  • Some callouts aren't being recognized, causing invalid xml
  • Some special characters such as asterisks need to be escaped, otherwise they cause valid xml (Asciidoctor: Handle ` as monospaced literal #531)
  • Beta, experimental, et all aren't showing up (Asciidoctor: Support added inline macro #548 + Asciidoctor: Remaining change admonishments #573 + Asciidoctor: Implement beta and experimental #662)
  • Include-tagged doesn't do anything
  • Images don't work (Asciidoctor: Copy referenced images #541)
  • [[float]]/chunking doesn't work (this might be the same as "Table of contents doesn't explode"?) (Asciidoctor: Fix raw xslt opts #527)
  • Broken definition lists cause a crash (Asciidoctor: Fix crash on broken definition lists #540)
  • Edit url is broken (Asciidoctor: Fix edit url for nested roots #546)
  • Callout icons aren't showing up (Asciidoctor: Copy callout icons #570)
  • Admonishments images aren't copied (Asciidoctor: Copy admonition images #604)
  • Kibana guide fails when copying images (Asciidoctor: Better handling for resources attribute #574)
  • OPEN IN CONSOLE isn't kicking in (Asciidoctor: Support for "open in widget" #627)
  • ID generation doesn't quite line up (What's Next -> _whats_next in asciidoctor vs _what_8217_s_next in asciidoc)
  • We've decided we're ok with this. We'll monitor when these IDs clash and see if they are broken, but for now all of the changes are good.
• Clean up the XML file after we're done with it (Asciidoctor: Drop xml file after docbook #547)
• Come up with some way of being sure that the rendered results are perfect or close enough. (Done with https://github.com/elastic/docs/blob/master/integtest/html_diff)

[bmorelli25]

I wasn't able to build the APM Server documentation due to a bunch of parser errors, but the agents look lovely! Two additional things I noticed:

• Images don't work
• [[float]]/chunking doesn't work (this might be the same as "Table of contents doesn't explode"?)

[nik9000]

I wasn't able to build the APM Server documentation due to a bunch of parser errors, but the agents look lovely! Two additional things I noticed:

I've added it to my checklist!

[nik9000]

Pinging @clintongormley who wanted to have a look at these.

My plan was to make some asciidoctor plugins to try and smooth out the differences that I've listed.

[lcawl]

Related to #567, I would like to use this asciidoctor functionality: https://asciidoctor.org/docs/asciidoc-writers-guide/#target-window-and-role-attributes-for-links
When I try it with the following test in the Stack Overview, however:

:linkattrs:
https://www.elastic.co/subscriptions["TEST", role="xpack"]

... it's not recognizing the role attribute. It outputs it as additional text:

[lcawl]

You might already be aware, but when I test the Kibana Guide via "docbldkb --asciidoctor --lenient", it fails as follows:

asciidoctor: WARNING: management/advanced-options.asciidoc: line 18: invalid style for paragraph: float

The above error disappears when I clean up the unnecessary float per elastic/kibana#29875

The following error persists, however:

/Users/lcawley/GitRepo/docs/resources/asciidoctor/lib/copy_images/extension.rb:50:in +': asciidoctor: FAILED: /Users/lcawley/GitRepo/kibana/docs/index.asciidoc: Failed to load AsciiDoc document - no implicit conversion of nil into Array (TypeError) from /Users/lcawley/GitRepo/docs/resources/asciidoctor/lib/copy_images/extension.rb:50:in find_source'
from /Users/lcawley/GitRepo/docs/resources/asciidoctor/lib/copy_images/extension.rb:24:in `process_block'

[lcawl]

Here's another issue that might be related to #541. I found it while working on elastic/stack-docs#197

The images that get added for special blocks (like IMPORTANT) are missing. For example, see:

When those are fixed, however, I think the Installation and Upgrade Guide is in a good state to start using Asciidoctor.

[nik9000]

When those are fixed, however, I think the Installation and Upgrade Guide is in a good state to start using Asciidoctor.

Nice! I did know about the issue but hadn't added it to the list. I don't really have an excuse. I just suck it in my head. Anyway. I'm glad to know that is working well there.

The following error persists, however

Yikes! That is in my code. I'll add it to the list as well.

[nik9000]

The following error persists, however:

@lcawl that error ought to be gone now that we have #570, but I've opened #574 to improve the error handling it to be double sure.

[lcawl]

The Stack Overview build (using the alias in #649) is failing now:

INFO:build_docs:/out/html_docs/index.xml:6534: element callout: validity error : Syntax of value for attribute arearefs of callout is not valid

Nevermind, I was able to debug the cause with the --lenient parameter. It's related to callouts and mismatched separators.

[lcawl]

I hadn't tested building the X-Pack Reference until now: #653

It's failing with the following errors:

Lisas-MBP-2:GitRepo lcawley$ docbldx --open --lenient
INFO:docker build:Building the docker image that will build the docs. Expect this to take somewhere between a hundred milliseconds and five minutes.
INFO:build_docs:Building HTML from /doc/x-pack/docs/en/index.asciidoc
INFO:build_docs:
INFO:build_docs:asciidoctor: ERROR: index.asciidoc: line 22: include file not found: /doc/elasticsearch-extra/x-pack-elasticsearch/docs/en/security/index.asciidoc
INFO:build_docs:asciidoctor: ERROR: index.asciidoc: line 26: include file not found: /doc/elasticsearch-extra/x-pack-elasticsearch/docs/en/watcher/index.asciidoc
INFO:build_docs:asciidoctor: ERROR: index.asciidoc: line 28: include file not found: /doc/elasticsearch-extra/x-pack-elasticsearch/docs/en/ml/index.asciidoc

Those errors remind me of the problems we had earlier with the --resource paths, but I can't find the relevant PR. Is it #601 ? Alternatively, they might be related to the fact that these x-pack-elasticsearch and x-pack-kibana repos exist under *-extra folders.

[nik9000]

Yeah no think it is the extra thing. I'll look at it. I expect it has to do with how I mount the repos for docker.

…

On Wed, Feb 27, 2019, 12:18 Lisa Cawley ***@***.***> wrote: I hadn't tested building the X-Pack Reference until now: #653 <#653> It's failing with the following errors: Lisas-MBP-2:GitRepo lcawley$ docbldx --open --lenient INFO:docker build:Building the docker image that will build the docs. Expect this to take somewhere between a hundred milliseconds and five minutes. INFO:build_docs:Building HTML from /doc/x-pack/docs/en/index.asciidoc INFO:build_docs: INFO:build_docs:asciidoctor: ERROR: index.asciidoc: line 22: include file not found: /doc/elasticsearch-extra/x-pack-elasticsearch/docs/en/security/index.asciidoc INFO:build_docs:asciidoctor: ERROR: index.asciidoc: line 26: include file not found: /doc/elasticsearch-extra/x-pack-elasticsearch/docs/en/watcher/index.asciidoc INFO:build_docs:asciidoctor: ERROR: index.asciidoc: line 28: include file not found: /doc/elasticsearch-extra/x-pack-elasticsearch/docs/en/ml/index.asciidoc Those errors remind me of the problems we had earlier with the --resource paths, but I can't find the relevant PR. Alternatively, they might be related to the fact that these x-pack-elasticsearch and x-pack-kibana repos exist under *-extra folders. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#505 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AANLouFJKrt01O25e6Tz5A0DNpCN1229ks5vRr3hgaJpZM4ZQQRi> .