[PATCH 0/9] Replace some mocking with with-http-server*, avoid hardcoding ports,
Maxime 5 years ago Maxime 5 years ago Maxime 5 years ago Maxime 5 years ago Maxime 5 years ago Maxime 5 years ago Maxime 5 years ago Maxime 5 years ago Maxime 5 years ago Maxime 5 years ago Ludovic 5 years ago Ludovic 5 years ago Maxime 5 years ago Maxime 5 years ago Maxime 5 years ago Maxime 5 years ago Maxime 5 years ago Ludovic 5 years ago Maxime 5 years ago Ludovic 5 years ago Maxime 5 years ago Ludovic 5 years ago Maxime 5 years ago Maxime 5 years ago Maxime 5 years ago [PATCH 0/9] Replace some mocking with with-http-server*, avoid hardcoding ports, Open Details 2 participants Ludovic Courtès Maxime Devos Owner unassigned Submitted by Maxime Devos Severity normal Debbugs page X-Debbugs-CC: ludo@gnu.org Hi, This patch series addresses: Ludovic Courtès schreef op zo 16-01-2022 om 23:19 [+0100]: Toggle quote (7 lines) > I think the whole point of having the ‘%github-api’ parameter is that > it allows us to mock the HTTP server instead of having to override > bindings such as ‘http-fetch’. > > I’d have a slight preference for doing that, similar to what is done > in tests/cpan.scm for instance.
WDYT? (see https://issues.guix.gnu.org/53060#3 .) by extending with-http-server to with-http-server* to allow arbitrary request handlers and extending with-http-server to allow verifying the URI of a request. tests/cpan.scm has been modified to verify the URIs. tests/import-github.scm and tests/minetest.scm have been modified to avoid mocking. Somewhat unrelated, tests/lint.scm, tests/cpan.scm and tests/challenge.scm have been modified to avoid hard coding ports, as a follow-up to commit c05ceaf2b650d090cf39a048193505cb4e6bd257: [...] Previously, test cases could fail if some process was listening at a hard-coded port.
This patch eliminates most of these potential failures, by automatically assigning an unbound port. This should allow for building multiple guix trees in parallel outside a build container, though this is currently untested. [...] After this patch series, there's to my knowledge only one instance of hardcoded ports remaining, in tests/lint.scm ("home-page: Connection refused"). This patch series is also available at https://notabug.org/maximed/guix-gnunet (branch more-precise-http-tests, commit f0a0303c17b5aa92493aea5a6d28183421b7cf44).
Greetings, Maxime. -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYelcpBccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7irMAPwNeTw/NNF7cXaGMZlTV+D1k3p6 xFhNYna2mBtbycMXDwEAkcl04r0yhSxeFWzFd39Wtq0MbnlmB1iBu48nwDIQoA4= =56SL -----END PGP SIGNATURE----- Fixes: https://issues.guix.gnu.org/53060#3 Unfortunately, for some unknown reason (a limitation of (guix tests http) perhaps?), parallelism causes ECONNREFUSED in tests but not in the wild, so 'par-map' has to be mocked for now. * tests/minetest.scm (call-with-packages): Avoid mocking by running an actual HTTP server. * guix/import/minetest.scm (par-map): Allow mocking the Minetest importer's use of par-map without impacting anything else.
Suggested-by: Ludovic Courtès <ludo@gnu.org> --- guix/import/minetest.scm | 5 ++- tests/minetest.scm | 82 ++++++++++++++++++++++++---------------- 2 files changed, 53 insertions(+), 34 deletions(-) Toggle diff (151 lines) diff --git a/guix/import/minetest.scm b/guix/import/minetest.scm index 3b2cdcdcac..3eab5f703f 100644 --- a/guix/import/minetest.scm +++ b/guix/import/minetest.scm @@ -19,7 +19,6 @@ (define-module (guix import minetest) #:use-module (ice-9 match) #:use-module (ice-9 receive) - #:use-module (ice-9 threads) #:use-module (ice-9 hash-table) #:use-module (srfi srfi-1) #:use-module (srfi srfi-2) @@ -69,6 +68,10 @@ (define (delete-cr text) (string-delete #\cr text)) +;; Mocked by tests. +(define par-map (@ (ice-9 threads) par-map)) +(set! par-map par-map) + ;;; diff --git a/tests/minetest.scm b/tests/minetest.scm index cbb9e83889..bdd8bd0645 100644 --- a/tests/minetest.scm +++ b/tests/minetest.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2021 Maxime Devos <maximedevos@telenet.be> +;;; Copyright © 2021, 2022 Maxime Devos <maximedevos@telenet.be> ;;; ;;; This file is part of GNU Guix. ;;; @@ -23,6 +23,7 @@ #:use-module (guix import minetest) #:use-module (guix import utils) #:use-module (guix tests) + #:use-module (guix tests http) #:use-module (guix packages) #:use-module (guix git-download) #:use-module ((gnu packages minetest) @@ -30,6 +31,9 @@ #:use-module ((gnu packages base) #:select (hello)) #:use-module (json) + #:use-module (web request) + #:use-module (web uri) + #:use-module (web client) #:use-module (ice-9 match) #:use-module (srfi srfi-1) #:use-module (srfi srfi-26) @@ -152,7 +156,7 @@ (invalidate-memoization! minetest->guix-package) (define (scm->json-port scm) (open-input-string (scm->json-string scm))) - (define (handle-package url requested-author requested-name . rest) + (define (handle-package subresource requested-author requested-name . rest) (define relevant-argument-list (any (lambda (argument-list) (apply (lambda* (#:key (author "Author") (name "foo") @@ -164,14 +168,15 @@ argument-lists)) (when (not relevant-argument-list) (error "the package ~a/~a should be irrelevant, but ~a is fetched" - requested-author requested-name url)) - (scm->json-port - (apply (match rest - (("") make-package-json) - (("dependencies" "") make-dependencies-json) - (("releases" "") make-releases-json) - (_ (error "TODO ~a" rest))) - relevant-argument-list))) + requested-author requested-name subresource)) + (define json (apply + (match rest + (("") make-package-json) + (("dependencies" "") make-dependencies-json) + (("releases" "") make-releases-json) + (_ (error "TODO ~a" rest))) + relevant-argument-list)) + (values '() (lambda (port) (scm->json json port)))) (define (handle-mod-search sort) ;; Produce search results, sorted by SORT in descending order. (define arguments->key @@ -191,29 +196,40 @@ ("name" . ,name) ("type" . ,type)))) (define argument-list->json (cut apply arguments->json <>)) - (scm->json-port - (list->vector (filter-map argument-list->json sorted-argument-lists)))) - (mock ((guix http-client) http-fetch - (lambda* (url #:key headers) - (unless (string-prefix?
"mock://api/packages/" url) - (error "the URL ~a should not be used" url)) - (define resource - (substring url (string-length "mock://api/packages/"))) - (define components (string-split resource #\/)) - (match components - ((author name . rest) - (apply handle-package url author name rest)) - (((? (cut string-prefix? "?type=mod&q=" <>) query)) - (handle-mod-search - (cond ((string-contains query "sort=score") "score") - ((string-contains query "sort=downloads") "downloads") - (#t (error "search query ~a has unknown sort key" - query))))) - (_ - (error "the URL ~a should have an author and name component" - url))))) - (parameterize ((%contentdb-api "mock://api/")) - (thunk)))) + (define json + (list->vector (filter-map argument-list->json sorted-argument-lists))) + (values '() + (lambda (port) (scm->json json port)))) + (with-http-server* + (lambda (request _) + (unless (eq? 'GET (request-method request)) + (error "wrong HTTP method")) + (define resource (uri-path (request-uri request))) + (unless (string-prefix?
"/api/packages/" resource) + (error "the resource ~a should not be used" resource)) + (define subresource + (substring resource (string-length "/api/packages/"))) + (define components (string-split subresource #\/)) + (match components + ((author name . rest) + (apply handle-package subresource author name rest)) + (("") + (let ((query (uri-query (request-uri request)))) + (handle-mod-search + (cond ((string-contains query "sort=score") "score") + ((string-contains query "sort=downloads") "downloads") + (#t (error "search query ~a has unknown sort key" + query)))))) + (_ + (error "the resource ~a should have an author and name component" + resource)))) + (parameterize ((%contentdb-api + (format #f "http://localhost:~a/api/" (%http-server-port))) + (cu
The rest of this page is on the live site.