summaryrefslogtreecommitdiff
path: root/src/rebar_prv_common_test.erl
Commit message (Collapse)AuthorAgeFilesLines
* Reload logger config in CT suitesFred Hebert2019-10-041-1/+1
| | | | | | | | | | | | | | | | | | | Fixes #2159; this is done by force-reloading the handlers to match the config. This can create a bit of a funny effect if sys.config specifies an INFO log level (or lower) is specified. While apps are booted for config changes before the cth_failonly hook is enabled, supervision and other application log messages can start being output. They will start being suppressed once the CT run begins. This is not a bug, it's a race in instantiation of control and enabling of log levels. Nothing we can do about that. It might however surprise people a good bit. If non-default stdout handlers are added, they are similarly likely to become noisy; specifying a test-specific sys.config file may be necessary then. Also includes a bump of cth_readable version, which now checks for updates to the log formatter on every test output.
* Add a --fail_fast switch to common testFred Hebert2018-12-171-4/+22
| | | | | | | | | | | | | | | | | | | | | | This switch aborts the rebar3 run after the first test failure is reported. Since Common Test does not support this feature and it can be useful for interactive development (a demand from one of the teams I work with), I decided to add the feature as an experimental CT hook. The hook notices any test failure, and forcefully aborts the current script execution, but only during the initialization phase of the next test; this ensure that all other hooks for a post-test failure have the time to do things right. The CT logs _will_ be interupted, and hooks of all kinds may also suffer. Since this might be a bit tricky to support internally, I will be fine with a review that results in "no, Fred. Please test this in a project plugin first" or if I get told to use profiles to mature the feature (i.e. `rebar3 as failfast ct ...`) with a custom hook declaration.
* Abstracted path managementFred Hebert2018-10-111-7/+5
| | | | | | | | | | | | | | | | Move path management out of rebar_utils manual code path function handling (which we leave there for backwards compat), and centralize them to allow easier coordination of paths between plugins and deps. On top of path handling, do a check of loaded modules to only purge and reload those that actually need it done in order to prevent all kinds of weird interaction and accidental purge kills. It also allows the possible cohabitation of both at once, with a "in case of conflict pick X" as a policy Changing path handling in providers also highlighted a bunch of bugs in some tests and appears to fix some in other providers, specifically around plugins.
* Logger support in cth_readable; compact CT outputFred Hebert2018-05-051-14/+20
| | | | | | | | | | | | | | | | | | | | | | | | cth_readable 1.4.0 supports the new logger interface from OTP-21, which likely breaks compatibility with R16 builds. It also includes a new compact interface, displaying output such as: ===> Running Common Test suites... %%% rebar_alias_SUITE: ...... %%% rebar_as_SUITE: ........... %%% rebar_compile_SUITE:................................ ............................ %%% rebar_compile_SUITE ==> test_name: SKIPPED %%% rebar_compile_SUITE ==> {tc_user_skip,"compile:env_compiler_options/0 available"} .. %%% rebar_cover_SUITE: ............. %%% rebar_ct_SUITE: .................................... Allowing to display more tests within less screen space. This mode has been added to the ct_readable option under the name 'compact' (now supporting true | false | compact), and has been made default for rebar3.
* add eunit and ct option to set coverdata file nameTristan Sloughter2017-12-041-3/+10
|
* Add experimental support for ct --retry optionFred Hebert2017-11-241-4/+19
| | | | | | | | | | | | | | | | | This commit adds a common test hook along with the cth_readable stuff whose role is to track failing test cases, and create a test specification out of them. The test specification is dumped on disk at _build/<profile>/logs/retry.spec and can be accessed by calling 'rebar3 ct --retry'. This will auto-load the spec file if it can be found and re-run the failing cases. If any other argument is found on the list specifying tests, the '--retry' argument is ignored. All code for this is marked as experimental in case we end up (keeping and then) dropping the feature.
* Revert "Add experimental support for ct --retry option"Fred Hebert2017-11-241-23/+5
|
* Add experimental support for ct --retry optionFred Hebert2017-11-231-5/+23
| | | | | | | | | | | | | | | | | This commit adds a common test hook along with the cth_readable stuff whose role is to track failing test cases, and create a test specification out of them. The test specification is dumped on disk at _build/<profile>/logs/retry.spec and can be accessed by calling 'rebar3 ct --retry'. This will auto-load the spec file if it can be found and re-run the failing cases. If any other argument is found on the list specifying tests, the '--retry' argument is ignored. All code for this is marked as experimental in case we end up (keeping and then) dropping the feature.
* OTP-21 readiness, Full Unicode supportFred Hebert2017-11-161-1/+1
| | | | | | | | | | | | This replaces all deprecated function usage by alternative ones based on a version switch enacted at compile time, preventing all warnings. This will likely introduce some possible runtime errors in using a Rebar3 compiled on OTP-20 or OTP-21 back in versions 19 and earlier, but we can't really work around that. A bunch of dependencies have been updated to support OTP-21 without warnings as well.
* Unicode support in all the placesFred Hebert2017-08-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This is done through 3 main change groups: - replacing `~s` by `~ts` in format strings, so that strings that contain unicode are properly printed rather than crashing - adding the `unicode` argument to all function of the `re` module to ensure transformations on strings containing unicode data are valid instead of crashing (see issue #1302) - replacing `ec_cnv:to_binary/1` and `ec_cnv:to_list/1` with matching functions in `rebar_utils`. The last point has been done, rather than modifying and updating erlware commons, because binary and list conversions can be a contentious subject. For example, if what is being handled is actually bytes from a given binary stream, then forcing a byte-oriented interpretation of the data can corrupt it. As such, it does not appear safe to modify erlware commons' conversion functions since it may not be safe for all its users. Instead, rebar3 reimplements a subset of them (only converting atoms and chardata, ignoring numbers) with the explicit purpose of handling unicode string data. Tests were left as unchanged as possible. This may impact the ability to run rebar3's own suites in a unicode path, but respects a principle of least change for such a large patch.
* add compile_only option to ct providerTristan Sloughter2017-07-131-2/+12
|
* Handle internal CT failuresFred Hebert2017-05-111-2/+9
| | | | | | | | | | | | | | This is based on issue #1517 where out of nowhere, CT has returned a user's error code directly. This in turn caused a crashdump in rebar3 itself. This patch handles the unexpected cases by: a) not trying to format them b) converting them to an error whenever they happen The execution flow is still interrupted, but we should fail with a clearer error than a crashdump.
* Drop R15 supportFred Hebert2017-05-031-3/+0
| | | | | | It can't even fetch packages from Hex anymore because of old SSL/TLS libraries, and so it can't bootstrap anymore either. Plus R20 comes out soon, and 5 major versions is quite enough.
* Fix typo, occured -> occurredLuis Rascao2017-04-121-1/+1
|
* Typo: seperate -> separateMichiel Beijen2017-03-271-8/+8
|
* Allow single test spec in ct_optsSiri Hansen2017-03-061-12/+15
| | | | | | The option {spec,Specs} is allowed in ct_opts, but rebar_prv_common_test:test_dirs did not take into account that Specs could also be a string only, i.e. not a list of strings.
* Handle errors from ct_testspecSiri Hansen2016-12-221-3/+5
|
* Make sure ct_testspec is loadedSiri Hansen2016-12-221-0/+1
| | | | ... before calling erlang:function_exported(ct_testspec,get_tests,1).
* Add all dirs from test specSiri Hansen2016-12-201-12/+69
| | | | | | | | | | Parse given test specs and add all spec- and suite directories as extra_src_dirs in order to ensure that all these directories are copied to the _build area and the suites are compiled. Specs located in the project- or app root are explicitly copied to the _build area in order to avoid recursive copying of the complete directory tree.
* Allow using relative path to suite in project rootSiri Hansen2016-12-131-14/+1
|
* Translate path to testspecSiri Hansen2016-12-091-33/+28
| | | | | | This is a bugfix. It makes sure that the given path to a testspec is translated so common_test will pick the spec from the _build directory, and not from the source tree.
* Add directory of testspec as extra_src_dirSiri Hansen2016-12-081-8/+8
| | | | | | This is necessary in order to automatically get the testspec included as an artifact (i.e. copied to the _build dir) in the case when it is stored in another directory than 'test'.
* Improve merge of command line options and config optionsSiri Hansen2016-12-071-25/+52
| | | | | | | | | | | | | Bug: option 'spec' is not specifically handled when merging options from the command line with options from rebar.config. Due to this, if the config specifies a 'spec', then this will take precedence over any 'dir' and/or 'suite' on the command line. This commit takes special care of all options that can be used to select tests - meaning that if any of the options 'spec', 'dir', 'suite', 'group' or 'case' are specified on the command line, then all 'spec', 'dir', 'suite', 'group' and 'case' options in rebar.config will be ignored.
* Fix rebar3 dialyzer warningsFred Hebert2016-11-191-11/+4
| | | | Some tricky changes in there but should be okay
* fix "helpful" compiler spelling correctionalisdair sullivan2016-10-171-1/+1
|
* allow test specifications to be passed via the command linealisdair sullivan2016-10-171-3/+8
| | | | | | `rebar3 ct --spec foo.spec,bar.spec,baz.spec` now works also added support for the `join_specs` flag on the command line
* Some post-review changes:Alexander Sedov2016-10-111-0/+4
| | | | | - restore path after loading applications, - helpful comments.
* Avoid backward-compatibility-breaking changes.Alexander Sedov2016-10-111-3/+3
|
* Made reading sys.configs consistent with OTP specification.Alexander Sedov2016-10-111-3/+3
|
* Made Common Test load the user's applications before slurping config.Alexander Sedov2016-10-111-0/+1
|
* Typo fix.Alexander Sedov2016-08-041-1/+1
|
* add support for passing a sys_config to common testTristan Sloughter2016-06-111-1/+24
|
* Add sname, name, setcookie option in eunit, ctsoranoba2016-04-071-1/+10
|
* supported dist_node in ct and eunitsoranoba2016-04-071-0/+5
|
* allow all `ct_opts` arguments to be passed through to `ct:run_test/1`alisdair sullivan2016-04-011-2/+4
|
* move definition of 'EUNIT' macro to eunit provideralisdair sullivan2016-03-031-1/+19
| | | | add definition of 'COMMON_TEST' macro to eunit provider
* Take CT options errors and turn them to warningsFred Hebert2016-03-021-6/+9
| | | | | | | | The idea is that given we accept arbitrary config items for CT, we should similarly be able to pass unsupported options and keep things running. However for unsupported options, a warning is very useful to have.
* Merge pull request #1099 from talentdeficit/ct_includeTristan Sloughter2016-03-011-14/+30
|\ | | | | add support for common tests `include` flag
| * add support for common tests `include` flagalisdair sullivan2016-03-011-14/+30
| |
* | Merge pull request #1098 from talentdeficit/ct_warn_on_cover_specTristan Sloughter2016-03-011-1/+3
|\ \ | | | | | | error on a cover spec in ct_opts
| * | add a link to the docs about `test_spec` and `cover` in ct warningsalisdair sullivan2016-03-011-2/+2
| | |
| * | error on a cover spec in ct_optsalisdair sullivan2016-03-011-0/+2
| |/
* | in providers that don't run per app run all available hooksTristan Sloughter2016-02-281-4/+5
|/
* fix paths for multiple app projects when running ct with no --dir argumentalisdair sullivan2016-01-301-7/+13
|
* don't strip the project apps when running `ct` with just a root suite specifiedalisdair sullivan2016-01-301-2/+3
| | | | this ensures the project apps are compiled to `lib/` instead of `extras/`
* put `extra' ct test suites in `extras/apps/APPNAME' rather than `extras'alisdair sullivan2015-12-201-1/+2
| | | | this allows repeated test suite names across apps without conflicts
* allow ct suites to be specified at the root of a project (or root of app)alisdair sullivan2015-12-201-2/+27
| | | | | | | | previously rebar3 dropped suites declared at the root of the project (via `--suite=whatever_SUITE' probably) and warned. this was because the compiler would recursively copy and compile everything in the directory indicated by the test suite. this changes the copy mechanism to only copy erl source files and directories that end with `_SUITE_data' into the `extras' dir in `_build'
* whitespace cleanupalisdair sullivan2015-12-201-1/+1
|
* error on ct/eunit argument errors instead of warningalisdair sullivan2015-11-151-42/+71
|
* warn on incorrectly specified test options in `rebar.config`alisdair sullivan2015-11-141-6/+21
| | | | | | | when `ct_opts`, `eunit_tests`, `eunit_first_files`, `ct_first_files`, `erl_first_files`, `eunit_compile_opts`, `ct_compile_opts` and `erl_opts` have values that are single non-list terms warn and try wrapping them in a list when processing them in the `eunit` and `ct` providers
generated by cgit v1.1 at 2024-10-07 04:28:26 +0000