aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeLines
...
| * | | | | | | | | | | | | | | Remove unnecessary use of partialGravatar Numerlor2021-02-05-6/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | run_in_executor can provide args to the func it's passed in, making the use of partial unnecessary. This will also make it more convenient to move to asyncio.to_thread when the codebase is switched to python 3.9
| * | | | | | | | | | | | | | | Avoid from import on _batch_parserGravatar Numerlor2021-02-03-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The tests import the modules the other way around causing a circular import
| * | | | | | | | | | | | | | | Use create_task utilGravatar Numerlor2021-02-03-2/+3
| | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | Create a task for setting the redis result instead of awaitingGravatar Numerlor2021-02-03-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The queue parsing doesn't depend on anything with redis, so the await only delays the result being set on the future.
| * | | | | | | | | | | | | | | Move BeautifulSoup parsing into an executorGravatar Numerlor2021-02-03-1/+4
| | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | Remove url lockGravatar Numerlor2021-02-03-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The items are added to the futures dict before a context switch can occur, making the subsequent requests to the url skip the queue extend and suspend at the future await
| * | | | | | | | | | | | | | | Use more consistent naming for doc items and symbolsGravatar Numerlor2021-02-03-71/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Trailing commas were also added where missing
| * | | | | | | | | | | | | | | Import module directly to avoid circular dependency and use of typeGravatar Numerlor2021-02-03-13/+11
| | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | Attempt to resolve first part of the argument in case of a failed lookupGravatar Numerlor2021-02-02-2/+12
| | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | Require a set of names to ignore instead of a blanket ignoringGravatar Numerlor2021-01-30-16/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As work is done on the modules the wrapper and wrapped functions are in more conflicts can occur, previously this could be missed as the info log that was done in case the error was suppressed was done when modules were being initialized during which there is a logging spam.
| * | | | | | | | | | | | | | | Cancel current task inside coro to not keep track of attempts manuallyGravatar Numerlor2021-01-23-8/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The scheduler shields the coroutine from cancellation so we can cancel the scheduler's tasks inside of it to avoid the error from multiple tasks with the same id trying to be scheduled which the manual tracking of attempts solved Co-authored-by: MarkKoz <[email protected]>
| * | | | | | | | | | | | | | | Wrap whole string in shortenGravatar Numerlor2021-01-23-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This helps avoid subtracting the length of "Moved: " from the shorten index Co-authored-by: MarkKoz <[email protected]>
| * | | | | | | | | | | | | | | Avoid errors when the first element is longer than the truncation indexGravatar Numerlor2021-01-23-3/+10
| | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | lock markdownify version to 0.6.1Gravatar Numerlor2021-01-23-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the 0.6.3 update brought a change that fails to ignore newlines in the html, introducing unnecessary lines into the output
| * | | | | | | | | | | | | | | Create a namespace varGravatar Numerlor2021-01-23-8/+9
| | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | Properly handle cache being clearedGravatar Numerlor2021-01-23-13/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously the code deleted the entry of all of the DocItems of the page after its contents were requested once, but this caused problems when the cache was cleared when it expired. Instead of deleting the entry to check if it should be queued on the next item request, we keep it and create an entry in the _item_futures dict for all items again and check for containment there. To avoid populating the queue multiple times with the same item in some cases the futures cleanup task will now only run when the queue is empty
| * | | | | | | | | | | | | | | Merge remote-tracking branch 'upstream/master' into doc-impGravatar Numerlor2021-01-22-323/+1314
| |\ \ \ \ \ \ \ \ \ \ \ \ \ \ \
| * | | | | | | | | | | | | | | | Fix issues with multiple gets being suspended when a refresh startsGravatar Numerlor2021-01-22-6/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With a normal event, if multiple gets were suspended and a refresh started, we'd continue the refresh after the first get finished and set the event which would be the same behaviour as the one it tried to fix. This is avoided by using a counter that's incremented every time a context manager is entered around an event and only setting the event when that counter reaches a zero after everything exited the context mgr
| * | | | | | | | | | | | | | | | Use inspect.unwrap instead of manually unwrappingGravatar Numerlor2021-01-21-3/+1
| | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | Use a defaultdict for item futuresGravatar Numerlor2021-01-15-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To be able to set the attribute at the start of the coro we need to be able to access the item's future before we know about all the other items. This also saves us from having to add them all as the queue parser or get_markdown will create the futures for us dynamically
| * | | | | | | | | | | | | | | | Prevent an inventory refresh while waiting for item cacheGravatar Numerlor2021-01-14-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If an inventory refresh was started while the symbol embed coroutine was suspended, it could cause the parser to try to fetch a non existent future if the markdown was requested after it was cleared but before new inventories were loaded in.
| * | | | | | | | | | | | | | | | Raise an error or log a warning if there's a global name conflictGravatar Numerlor2021-01-14-5/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When wrapper uses a global name, which conflicts with a global name from wrapped's module that wrapped uses for its annotations, we run into a situation that can't be solved without changing one of the names, so an error is raised to give this clearer meaning. The check may be erroneous in some edge cases or the objects the conflicting names refer to can be functionally identical, so the error can be turned into a logged warning.
| * | | | | | | | | | | | | | | | Ensure footer is actually max 100 charsGravatar Numerlor2021-01-14-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Shortening the renamed symbols string to 100 chars is not accurate as the footer also contains a string before that, subtracting its length fixes this.
| * | | | | | | | | | | | | | | | Set the user_requested attribute at the start of the coroutineGravatar Numerlor2021-01-14-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A context switch may occur when we're waiting for the web page response, during which a clear could be triggered. If the event is not set before that we could end up with the dictionary changing sizes, or if a copy was made, a future that'd never finish as it'd be cleared from the queue and the futures dict
| * | | | | | | | | | | | | | | | Reuse the redis key instead of creating a new string for the expires setGravatar Numerlor2021-01-14-2/+2
| | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | Change typehint name casing to PascalCaseGravatar Numerlor2021-01-14-8/+9
| | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | Keep trakck of the init task and cancel it when the cog is unloadedGravatar Numerlor2021-01-14-1/+2
| | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | Simplify control flowGravatar Numerlor2021-01-14-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Co-authored-by: MarkKoz <[email protected]>
| * | | | | | | | | | | | | | | | Use different task ids for every inventory reschedule attemptsGravatar Numerlor2021-01-14-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The scheduler can't keep track of multiple tasks with the same id, and rescheduling the update task using the same id within an already scheduled update task caused the new task to get ignored as the old task only got deleted from the scheduler after it was finished
| * | | | | | | | | | | | | | | | Correct return type annotationGravatar Numerlor2021-01-14-1/+1
| | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | Turn update_single into a normal functionGravatar Numerlor2021-01-14-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The method no longer runs anything asynchronous
| * | | | | | | | | | | | | | | | Fix docstringGravatar Numerlor2021-01-11-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The decorator works in revers to what the docstring explained
| * | | | | | | | | | | | | | | | Return whitespace to its previous stateGravatar Numerlor2021-01-11-1/+1
| | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | Make REFRESH_EVENT an instance variableGravatar Numerlor2021-01-11-6/+6
| | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | Merge remote-tracking branch 'upstream/master' into doc-impGravatar Numerlor2021-01-11-1234/+646
| |\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \
| * | | | | | | | | | | | | | | | | Remove old reference to CachedParser and unused constGravatar Numerlor2021-01-11-4/+1
| | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | Check for containment instead of always getting the value from the dictGravatar Numerlor2021-01-11-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Getting the value from a defaultdict will always create the key for it, creating unnecessary entries every time a symbol is fetched from the bot
| * | | | | | | | | | | | | | | | | Notify the user that inventories were refreshed on cache clearsGravatar Numerlor2021-01-11-1/+1
| | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | Don't convert package names into lowercaseGravatar Numerlor2021-01-11-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The converter used to set them already ensures this for us, making the call redundant
| * | | | | | | | | | | | | | | | | Ensure footer fits into messageGravatar Numerlor2021-01-11-3/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The footer also now says Moved: at the start to clarify the meaning of the symbols to the user
| * | | | | | | | | | | | | | | | | Use a dictionary of lists instead of set for renamed symbolsGravatar Numerlor2021-01-11-8/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A dictionary allows us to grab the original symbol name and then get all the renamed symbols from it, with the improvements to `ensure_unique_symbol_name` we can also use lists instead of sets as each symbol we add should be unique
| * | | | | | | | | | | | | | | | | Handle renaming conflicting symbols in ensure_unique_symbol_nameGravatar Numerlor2021-01-11-15/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously update_single looped this function until there were no duplicates and when creating new symbols the function had to check if the symbol to create a new name from started with a group/package to avoid redundancy. The new approach ensures a new symbol is always unique when returning by handling the containment check inside and outputting a symbol name in the format of package.group.symbol which should always be unique
| * | | | | | | | | | | | | | | | | Update outdated docstringGravatar Numerlor2021-01-11-1/+1
| | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | Send a message to devlog instead of logging a warningGravatar Numerlor2021-01-11-1/+30
| | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | Resolve wrapped command callbacks in the source commandGravatar Numerlor2021-01-10-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Without this the command will fetch the source of the wrapper
| * | | | | | | | | | | | | | | | | Create decorator for update_wrapper_globals mimicking functools.wrapsGravatar Numerlor2021-01-10-9/+30
| | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | Change the func name to wrapped for clarityGravatar Numerlor2021-01-10-6/+6
| | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | Rename CachedParser to BatchParser and move it to its own moduleGravatar Numerlor2021-01-10-166/+180
| | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | Defer import to avoid circular importsGravatar Numerlor2021-01-10-1/+1
| | | | | | | | | | | | | | | | | |
| * | | | | | | | | | | | | | | | | Move functions strictly related to parsing html to the _html moduleGravatar Numerlor2021-01-10-120/+126
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some constants need to be shared between html and parsing, because they may also be wanted to be edited by the cog user to change the behaviour, they were moved into the package's init.