Purpose
-------

The purpose of these changes is to decouple the LTI 1.3 launch from the LtiConsumerXBlock. It is in accordance with the ADR "0007 Decouple LTI 1.3 Launch from XBlock and edX Platform", which is currently under review. The pull request for the ADR is here: https://github.com/openedx/xblock-lti-consumer/pull/281.

The general premise of these changes is to shift the responsibility of defining key launch claims to users of the library. Such claims include user ID, user role, resource link ID, etc. Prior to this change, this context was defined directly in the launch view by referencing XBlock fields and functions, thereby tying the LTI 1.3 launch to the XBlock. By shifting the responsibility out of the view, we will be able to genericize the launch and make it functional in more contexts than just the XBlock and the XBlock runtime.

In short, the key launch claims are encoded in an instance of a data class Lti1p3LaunchData. Users of the library will instantiate this class with necessary launch data to it and pass the instance to various methods of the Python API to communicate the data to the library. Please see the aforementioned ADR for more details about this decoupling strategy.

Note that the majority of these changes affect only the basic LTI 1.3 launch. There have largely been no changes to LTI 1.3 Advantage Services. The one exception is the Deep Linking content launch endpoint. This is because this launch is implemented in the basic LTI 1.3 launch, and it was necessary to make the same changes to the deep linking content launch to ensure that it works properly. Otherwise, LTI 1.3 Advantage Services are out of scope of these changes.

Change Summary for Developers
-----------------------------

Below is a summary of changes contained in this pull request.

* added an Lti1p3LaunchData data class
* added caching for Lti1p3LaunchData to limit data sent in request query or form parameters
* BREAKING CHANGE: modified Python API methods to take Lti1p3LaunchData as a required argument
** get_lti_1p3_launch_info
** get_lti_1p3_launch_start_url
** get_lti_1p3_content_url
* replaced references to LtiConsumerXBlock.location with Lti1p3LaunchData.config_id
* removed definition of key LTI 1.3 claims from the launch_gate_endpoint and instantiated Lti1p3LaunchData from within the LtiConsumerXBlock instead
* added a required launch_data_key request query parameter to the deep_linking_content_endpoint and refactored associated templates and template tags to pass this parameter in the request to the view

Change Summary for Course Staff and Instructors
-----------------------------------------------

The only changes relevant for course staff and instructors is that the access token and keyset URLs displayed in Studio have changed in format.

The old format was:

Access Token URL: https://courses.edx.org/api/lti_consumer/v1/token/block-v1:edX+999+2022Q3+type@lti_consumer+block@714c10a5e4df452da9d058788acb56be
Keyset URL: https://courses.edx.org/api/lti_consumer/v1/public_keysets/block-v1:edX+999+2022Q3+type@lti_consumer+block@714c10a5e4df452da9d058788acb56be

The new format is:

Access Token URL: https://courses.edx.org/api/lti_consumer/v1/token/c3f6af60-dbf2-4f85-8974-4ff870068d43
Keyset URL: https://courses.edx.org/api/lti_consumer/v1/public_keysets/c3f6af60-dbf2-4f85-8974-4ff870068d43

The difference is in the slug at the end of the URL. In the old format, the slug was the UsageKey of the XBlock associated with the LTI integration. In the new format, the slug is the config_id of the LtiConfiguration associated with the LTI integration. This is an iterative step toward decoupling the access_token_endpoint and the public_keyset_endpoint views from the XBlock location field. The XBlock location field appears as the usage_key parameter to both views. We cannot simply remove the usage_key parameter from the views, because existing LTI 1.3 integrations may have been created using the old format, and we need to maintain backwards compatibility. This change, however, prevents new integrations from being created that are coupled to the XBlock. In the future, we may address integrations that use the old format to fully decouple the XBlock from the views.

Testing
-------

Unit tests were added for all changes.

In addition, manual testing was performed using the instructions in the documents listed below.

* https://github.com/openedx/xblock-lti-consumer#lti-13
* https://openedx.atlassian.net/wiki/spaces/COMM/pages/1858601008/How+to+run+the+LTI+Validation+test

Resources
---------
JIRA: MST-1603: https://2u-internal.atlassian.net/browse/MST-1603

BREAKING CHANGE

* chore: Updating Python Requirements

* fix: update is_valid arg

Co-authored-by: Alie Langston <alangsto@wellesley.edu>

This should add this repo as a subcomponent of:

  https://backstage.openedx.org/catalog/default/component/LTI

Python Requirements Update

docs: Update catalog-info.yaml

* Remove an unnecessary annotation
* Add a comment back to the reference file and relevant OEP

Backstage will automatically pick up `yaml` files but not `yml` files.

* docs: readme badges and  refactor
* feat: add backstage catalog-info

Handle LtiError Error During LTI 1.1 Launch When Calling user_id for Unauthenticated User

In the LTI 1.1 launch handler, we set the user context, including the user_id. We do this by calling to the LMS's DjangoXBlockUserService to get information about the user. Sometimes, the user is unauthenticated. Sometimes, this is because the user is a web crawler. Other times, the user is a real user, but we do not know why the user is unauthenticated. We have some theories, but we have been unable to confirm them. Regardless, we should not surface a 500 error to the user.

This commit adds handling for the LtiError that is raised when a user is unauthenticated during an LTI 1.1 launch. It catches the LtiError and renders an error page. The error page that was used for LTI 1.3 launches, formerly named "lti_1p3_launch_error.html", has been renamed to "lti_launch_error.html" to reflect the fact that it is used for both LTI 1.1 and 1.3 launches. It was modified to remove the reference to the version of LTI used by the XBlock; these details are unnecessary for a learner, and removing them allows us to reuse a single template for both LTI versions.

This reverts commit 617d8781.

We have determined an appropriate course of action for these errors, so the additional logging is no longer necessary and should be removed.

Python Requirements Update

Move XBlock endpoints to Django models and implement backwards compatible views.

Relevant commits:
* refactor: move LTI 1.3 access token endpoint to plugin view
* refactor: remove the xblock handler and add tests to api view
* refactor: move the lti_1p3_launch_callback logic to the django view
* feat: adds access token view for backward compatibility
* refactor: make launch urls use config_id when block is missing
* refactor: remove launch_callback_handler from XBlock

Python Requirements Update

fix: missing exception details in logs

The error handler in LtiConsumerXBlock.lti_1p3_launch_callback logs a warning when a select set of exceptions are handled. That log does not contain useful information about the nature of the exception, because the exceptions were not being instantiated with error messages. The try...catch is a large block that contains code that can raise a multitude of errors, so these changes will enable better debugging.

This commit:
* adds helpful messages to the raised exceptions.
* adds the "exc_info=True" argument to include the stack trace of the handled exception.
* adds ValueError and TypeError to the list of handled exceptions, because the code can raise exceptions of these types.

test: add debugging log statements for user_id LtiError

This commit adds supplemental logging to diagnose the bug reported in MST-1540: https://2u-internal.atlassian.net/browse/MST-1540. The bug is that learners are encountering the LtiError when trying to do an LTI launch. The learners appear to be authenticated, so this error should not occur. The bug is not easily reproducible in production or development, so this supplemental logging is added to help understand the user's state when the error is raised.

The current hypothesis is that user is temporarily represented by the AnonymousUser in the request that is made when doing the LTI launch, despite the user otherwise being authenticated. Logging in Splunk suggests that this is the case, because logs are of the following form, "2022-07-22 15:10:14,214 ERROR 5067 [django.request] [user None] [ip <ip>] log.py:224 - Internal Server Error: /courses/<course_key>/xblock/<usage_key>/handler/lti_launch_handler", where the "user" is "None". This logging should prove or disprove this hypothesis and provide direction about where else to look.

This logging should be removed once MST-1540 is resolved.

fix: hidden lti_version and 1.3 fields in edit view and incorrect menu behavior in Javascript

This commit fixes three bugs.

1. The first bug is that the lti_version field is inappropriately hidden in the Studio author view edit menu when the selected config_type is database.

2. The second bug is that the editable_fields property of the LtiConsumerXBlock is inappropriately excluding LTI 1.3 fields when the config_type is database. The editable_fields property should include LTI 1.3 fields even when the config_type is database, because the Javascript defined in xblock_studio_view.js may want to show these fields if the user selects a different config_type in the menu. We want to support a dynamic edit menu, so these fields must be considered editable by the XBlock in order for the Javascript to be able to manipulate them.

3. The third bug is in inconsistent rendering of the Studio author view edit menu. Depending on the order in which a user selects lti_version, config_type, or lti_1p3_tool_key_mode, different sets of fields are displayed, due to the overlapping sets of rules that govern what fields should be hidden or shown for a given field selection. This commit corrects this inconsistent rendering by first showing all fields and then gradually hiding fields depending on the sets of rules, for each change to the fields.

feat: Add core LTI 1.3  and LTI Advantage configuration to LTIConfiguration model and support xBlock using the database

This commit adds additional core LTI 1.3 and LTI Advantage variables to the LTIConfiguration model. The additional core LTI 1.3 variables are lti_1p3_oidc_url, lti_1p3_launch_url, lti_1p3_tool_public_key, and lti_1p3_tool_keyset_url. The additional LTI Advantage variables are lti_advantage_enable_nrps, lti_advantage_deep_linking_enabled, lti_advantage_deep_linking_launch_url, and lti_advantage_ags_mode.

This commit also adds a configuration type to the LtiConsumerXBlock to support the storage of these LTI variables on the LTIConfiguration model (i.e. the database) instead of the xBlock itself.

Changes that allow the use of this configuration option are behind the lti_consumer.enable_database_config CourseWaffleFlag.

Modified the Lti1p3ApiAuthentication authentication backend to return
AnonymousUser instead of None. This allows DarkLangMiddleware to work
properly without crashing when middleware tries to get a user info
from the request.

This investigation and patch was originally done by @OlhaShyliaieva in:
  https://github.com/openedx/xblock-lti-consumer/pull/228

I'm just rebasing it onto the latest version to land this change.

Co-authored-by: Muhammad Soban Javed <58461728+iamsobanjaved@users.noreply.github.com>
Co-authored-by: Usama Sadiq <usama.sadiq@arbisoft.com>

* feat: add event tracking to lti launch

* style: move tracking fn to new file

* fix: add missed track call to LTI1.3

* feat: release tasks

* chore: Updating Python Requirements

* fix: downgrade boto3

Co-authored-by: Alie Langston <alangsto@wellesley.edu>

Python Requirements Update

fix: upgrade pip-tools to fix bug in versions 6.6.0 and 6.6.1

This commit upgrades the version of pip-tools used in this repository from 6.6.0 to 6.6.2.

In version 6.6.0 of pip-tools, there is a bug that is preventing pip-tools from working. This is breaking the Python requirements update GitHub action in this repository.

The error is "ImportError: cannot import name 'BAR_TYPES' from 'pip._internal.cli.progress_bars'". The error was reported here: https://github.com/jazzband/pip-tools/issues/1617. The fix to this bug was released in version 6.6.2 of pip-tools. See the comment here: https://github.com/jazzband/pip-tools/issues/1617#issuecomment-1126245586.

Version 6.6.1 of pip-tools also has a bug, which is fixed in version 6.6.2. I observed this issue breaking the Python requirements update GitHub action in another repository, so I have upgrade the version straight to 6.6.2. The issue in version 6.6.1 was reported here: https://github.com/jazzband/pip-tools/pull/1624.

We are removing this runtime attribute in openedx/edx-platform#30450.