aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeLines
...
| | * | | | | | | | | Add package in front of symbol as default fallbackGravatar Numerlor2020-11-10-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously weo nly added the package name for symbols that shared are named name with an another symbol, but in some edge cases we can get to this point with symbols that weren't renamed but have name conflicts, causing some to get overwritten completely without the capturing condition
| | * | | | | | | | | Cache user fetched symbols through redis.Gravatar Numerlor2020-11-10-2/+43
| | | | | | | | | | |
| | * | | | | | | | | Handle escaped backslashes in stringsGravatar Numerlor2020-10-18-3/+3
| | | | | | | | | | |
| | * | | | | | | | | Exclude headerlinks outside of current sectionGravatar Numerlor2020-10-10-9/+13
| | | | | | | | | | |
| | * | | | | | | | | Use more descriptive name for end_tag_filterGravatar Numerlor2020-10-10-5/+5
| | | | | | | | | | |
| | * | | | | | | | | Handle non dt fallback together with modulesGravatar Numerlor2020-10-10-6/+1
| | | | | | | | | | |
| | * | | | | | | | | Merge remote-tracking branch 'upstream/master' into doc-impGravatar Numerlor2020-10-10-3617/+2635
| | |\ \ \ \ \ \ \ \ \
| | * | | | | | | | | | Reschedule failed inventory updatesGravatar Numerlor2020-10-10-4/+35
| | | | | | | | | | | |
| | * | | | | | | | | | Remove sphinx and requests from PipfileGravatar Numerlor2020-10-10-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With our own implementation of sphinx's inventory fetching we no longer need the sphinx package, and requests which were used inside of it.
| | * | | | | | | | | | Use new async inventory fetchingGravatar Numerlor2020-10-10-64/+7
| | | | | | | | | | | |
| | * | | | | | | | | | Handle errors on inventory fetchingGravatar Numerlor2020-10-10-2/+35
| | | | | | | | | | | |
| | * | | | | | | | | | Add async implementation of sphinx fetch_inventoryGravatar Numerlor2020-10-10-0/+117
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The sphinx version of the function does a lot of checks that are unnecessary for the bot because it's not working with anything else related to docs. The custom implementation means we can throw some of the code out and get rid of sphinx as a dependency.
| | * | | | | | | | | | Add parentheses for clarityGravatar Numerlor2020-10-10-1/+1
| | | | | | | | | | | |
| | * | | | | | | | | | Make amount of included signatures configurableGravatar Numerlor2020-10-10-7/+10
| | | | | | | | | | | |
| | * | | | | | | | | | Collapse signatures between args instead of spacesGravatar Numerlor2020-10-10-3/+92
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The signature length needed more logic and shorter limits to ensure messages would fit in a discord message in a nice way.
| | * | | | | | | | | | Use List typehint that has a narrower scopeGravatar Numerlor2020-09-21-1/+1
| | | | | | | | | | | |
| | * | | | | | | | | | Handle cases with outdated bot inventories.Gravatar Numerlor2020-09-21-0/+3
| | | | | | | | | | | |
| | * | | | | | | | | | Use f strings instead of c style on copied codeGravatar Numerlor2020-09-21-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code copied over from MarkdownConverter's implementation used c style string formatting, there is no reason to keep the style of strings in our code
| | * | | | | | | | | | Properly truncate description markdownGravatar Numerlor2020-09-21-44/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous truncating implementation used a naive method that disregarded the actual markdown formatting, possibly resulting in it getting cut out. With the introduction of proper href tags this became impossible to manage without writing an actual parser; so the process was moved to happen when the gathered bs4 elements are being converted into markdown
| | * | | | | | | | | | Log exceptions from parsing taskGravatar Numerlor2020-09-20-3/+7
| | | | | | | | | | | |
| | * | | | | | | | | | Merge remote-tracking branch 'upstream/master' into doc-impGravatar Numerlor2020-09-20-2369/+6238
| | |\ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | # Conflicts: # bot/cogs/doc/cog.py # bot/converters.py # tests/bot/test_converters.py
| | * | | | | | | | | | | Remove unused cacheGravatar Numerlor2020-09-20-32/+0
| | | | | | | | | | | | |
| | * | | | | | | | | | | Replace forward ref with future annotations importGravatar Numerlor2020-09-20-1/+3
| | | | | | | | | | | | |
| | * | | | | | | | | | | Replace caching of soups with new class.Gravatar Numerlor2020-09-20-39/+119
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Storing BeautifulSoup objects could lead to memory problems because of their large footprint, the new class replaces the long term storage by parsing all items on the first fetch of the page and only storing their markdown string.
| | * | | | | | | | | | | Don't return fragment in DocItem urlGravatar Numerlor2020-09-16-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The fragment is only needed for the user and required sparingly returning only the url while keeping the fragment behind symbol_id simplifies the uses of the url without it.
| | * | | | | | | | | | | Remove old commentGravatar Numerlor2020-09-16-1/+0
| | | | | | | | | | | | |
| | * | | | | | | | | | | Properly parse labels add fallback for non dt tagsGravatar Numerlor2020-07-26-5/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Labels point to tags that aren't in description lists, like modules or doc symbols which we already handle. If by chance we get a symbol that we don't have in the group for general parsing and which isn't a dt tag, log it and don't attempt to parse signature and use general description parsing instead of parsing a dd tag.
| | * | | | | | | | | | | Rename description functions to be more generalGravatar Numerlor2020-07-26-8/+8
| | | | | | | | | | | | |
| | * | | | | | | | | | | Add more loggingGravatar Numerlor2020-07-26-0/+4
| | | | | | | | | | | | |
| | * | | | | | | | | | | Move MarkdownConverter subclass to separate moduleGravatar Numerlor2020-07-26-57/+60
| | | | | | | | | | | | |
| | * | | | | | | | | | | Fix markdownify's handling of h tags.Gravatar Numerlor2020-07-26-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Discord only allows `**` for bolding while the markdown from the default MarkdownConverter tries to use # time n with h*n* tags for different font weights.
| | * | | | | | | | | | | Parse NavigableStrings in symbol descriptions.Gravatar Numerlor2020-07-26-14/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a symbol, such as [term.numpy](https://matplotlib.org/3.1.1/glossary/index.html#term-numpy) had NavigableStrings as direct children, they were not included as bs4's SoupStrainer won't include both strings and tags in its filters. The implementation goes around the limitation by introducing a new optional flag, bypassing the default check which skips matching tags when the `text` argument is present.
| | * | | | | | | | | | | Ensure all renamed symbols are keptGravatar Numerlor2020-07-25-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After the restructure behaviour change in d790c404ca3dba3843f351d6f42e766956aa73a1, the add to renamed_symbols was not readded and symbols that only passed the first check were being missed.
| | * | | | | | | | | | | Change ValidPythonIdentifier tests to PackageName.Gravatar Numerlor2020-07-22-11/+10
| | | | | | | | | | | | |
| | * | | | | | | | | | | Remove paragraph chars from descriptionsGravatar Numerlor2020-07-22-1/+1
| | | | | | | | | | | | |
| | * | | | | | | | | | | Hold url parts in DocItem separately.Gravatar Numerlor2020-07-22-3/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows us to save up some memory by not creating unique strings with the base url repeated between them.
| | * | | | | | | | | | | Remove caching from get_symbol_embed.Gravatar Numerlor2020-07-22-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The web request is already cached, and parsing doesn't much more time, but without moving the logic around the cache prevents the stat increase when a symbol is requested.
| | * | | | | | | | | | | Change DocCog to use the new parsing module fully.Gravatar Numerlor2020-07-22-62/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The parsing module provides an interface for fetching the markdown from the symbol data provided to it. Because it's now fully done in an another module we can remove the needless parts from the cog.
| | * | | | | | | | | | | Create the parsing interface function.Gravatar Numerlor2020-07-22-21/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Other functions from the module are not intended to be used directly, with the interface of it being the added function which accepts the symbol and calls internals. All other names except imports and log had the underscore prefix added to accommodate this.
| | * | | | | | | | | | | Create a function for getting the result markdown.Gravatar Numerlor2020-07-22-0/+21
| | | | | | | | | | | | |
| | * | | | | | | | | | | Fix ordered list indices in markdown converter.Gravatar Numerlor2020-07-22-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | markdownify relies on the parent tag's index method, which goes through all of its contents, if there is anything else in the contents apart from the li tags, those indices are then shifted.
| | * | | | | | | | | | | Move DocMarkdownConverter to parsing.Gravatar Numerlor2020-07-22-34/+34
| | | | | | | | | | | | |
| | * | | | | | | | | | | Fix handling of elements when fetching signatures.Gravatar Numerlor2020-07-22-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After the change to `find_elements_until_tag`, the text contentsneed to be extracted from the tags instead of passing them directly to re.
| | * | | | | | | | | | | Remove conversion to str when finding elements.Gravatar Numerlor2020-07-22-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The tags need to be processed down the line, which is not viable on strings.
| | * | | | | | | | | | | Create a function for collecting signatures.Gravatar Numerlor2020-07-20-36/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By getting the signatures without the description we get more flexibility of parsing different symbol groups and decouple the logic from the description which can be parsed directly with the new `find_elements_until_tag` based function.
| | * | | | | | | | | | | Simplify module parsing method.Gravatar Numerlor2020-07-20-9/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of returning None and multiple values, the method now only returns the string of the description. Previously the parsing returned None and quit when appropriate tags for shortening the description were not found, but the new implementation simply defaults to the provided start tag if a better alternative is not found.
| | * | | | | | | | | | | Add function for finding tags until a matching tagGravatar Numerlor2020-07-20-0/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will allow flexibility in the future when collecting tags for the description and signature of symbols. The base is a function which accepts a callable which is called and iterated over, but 3 names with a partial function that has the callable supplied are provided to keep the outside interface neater.
| | * | | | | | | | | | | Simplify cutoff text.Gravatar Numerlor2020-07-20-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "read more" seemed out of place with no permalink over it.
| | * | | | | | | | | | | Remove permalink from truncated markdown.Gravatar Numerlor2020-07-20-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The permalink serves no functional purpose in the embed, as it is already included in the title. But it does add the complexity of passing in the url to the parser.
| | * | | | | | | | | | | Rename parser.py to parsing.py.Gravatar Numerlor2020-07-20-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Parser is a stdlib module name, a rename avoids shadowing it.