Conversation
e2c4216 to
f30b1f3
Compare
|
@bettio it seems that CI sometimes fails because it cannot find |
src/libAtomVM/term.h
Outdated
| #define BOXED_FUN_SIZE 3 | ||
| #define FLOAT_SIZE (sizeof(float_term_t) / sizeof(term) + 1) | ||
| #define REF_SIZE ((int) ((sizeof(uint64_t) / sizeof(term)) + 1)) | ||
| #define TERM_BOXED_PID_REF_SIZE (REF_SIZE + 1) |
There was a problem hiding this comment.
For consistency with the work we are doing lately I suggest renaming all PID_REF and pid_reference to PROCESS_REF and process_reference
src/libAtomVM/term.h
Outdated
| term *boxed_value = memory_heap_alloc(heap, TERM_BOXED_PID_REF_SIZE); | ||
| boxed_value[0] = TERM_BOXED_PID_REF_HEADER; | ||
|
|
||
| #if TERM_BYTES == 8 |
There was a problem hiding this comment.
nitpicking/very low priority comment: just for consistency I would suggest moving first the 4 branch:
#if TERM_BYTES == 4 ... #elif TERM_BYTES == 8.
Same applies to the other functions as well.
src/libAtomVM/term.c
Outdated
| return ret; | ||
|
|
||
| } else if (term_is_pid_reference(t)) { | ||
| int32_t process_id = term_pid_ref_to_process_id(t); |
There was a problem hiding this comment.
We are using uint32_t for process id.
There was a problem hiding this comment.
Are you sure? It's int32 here: https://github.com/atomvm/AtomVM/blob/main/src/libAtomVM/context.h#L106 and in many other places
tests/erlang_tests/test_monitor.erl
Outdated
| ok = test_alias(), | ||
| ok = test_monitor_alias(), | ||
| ok = test_monitor_alias_explicit_unalias(), | ||
| ok = test_monitor_down_alias(), |
There was a problem hiding this comment.
We should also test unhappy paths, such as:
- double/triple unalias
- alias a dead process
- unalias a dead process
In addition multiple aliases should be tested, this is legit but is should be properly tested.
src/libAtomVM/nifs.c
Outdated
| RAISE_ERROR(BADARG_ATOM); | ||
| } | ||
|
|
||
| while (!term_is_nil(options)) { |
There was a problem hiding this comment.
We should use term_is_nonempty_list to avoid issues with non proper lists.
src/libAtomVM/nifs.c
Outdated
| RAISE_ERROR(UNSUPPORTED_ATOM); | ||
| } else { | ||
| RAISE_ERROR(BADARG_ATOM); | ||
| } |
There was a problem hiding this comment.
here we check that last list item is nil.
src/libAtomVM/context.h
Outdated
| enum ContextMonitorAliasType | ||
| { | ||
| CONTEXT_MONITOR_ALIAS_EXPLICIT_UNALIAS, | ||
| CONTEXT_MONITOR_ALIAS_DEMONITOR, |
There was a problem hiding this comment.
Let's use PascalCase. See AVMCCS-N004 rule in our coding style guide.
(There is still some old code that has to be migrated to updated style).
4b23228 to
07373d0
Compare
|
@bettio added more tests and missing features. The CI is failing, but it seems unrelated to the changes |
bettio
left a comment
There was a problem hiding this comment.
Looks mostly good: there is only one change related to IS_NULL_PTR that is required. I suggest also rebasing on top of main, since main has a lot of fixes for flaky tests.
src/libAtomVM/opcodesswitch.h
Outdated
| Context *p = globalcontext_get_process_lock(glb, local_process_id); | ||
| if (p) { | ||
| struct MonitorAlias *alias = context_find_alias(p, ref_ticks); | ||
| if (!IS_NULL_PTR(alias)) { |
There was a problem hiding this comment.
We should not use !IS_NULL_PTR (x) since it is a shorthand for UNLIKELY(x == NULL). So we must use it only when it is unlikely that the pointer is null (that is mostly for error handling purposes).
There was a problem hiding this comment.
Done, but there are many occurrences of !IS_NULL_PTR in the code
92db4b7 to
1bdef14
Compare
|
esp32 tests are crashing |
pguyot
left a comment
There was a problem hiding this comment.
My main concern so far is that monitors are now extended to 2 or 3 words instead of 1 or 2. This may break some code that took monitors and passed it around as references, as the term_to_ref_ticks/term_from_ref_ticks is a common pattern. A similar breakage existed with resources being larger references if one tries to pass a resource where a 1/2 words ref is expected, but it is less likely as resources were not references so far. This may be the crash cause of esp32 tests.
port.erl
call(Port, Message, Timeout) ->
MonitorRef = monitor(port, Port),
Port ! {'$call', {self(), MonitorRef}, Message},
Result =
receive
{'DOWN', MonitorRef, port, Port, normal} ->
{error, noproc};
{'DOWN', MonitorRef, port, Port, Reason} ->
{error, Reason};
out_of_memory ->
out_of_memory;
{MonitorRef, Ret} ->
Ret
after Timeout ->
{error, timeout}
end,
demonitor(MonitorRef, [flush]),
Result.
uart_driver.c:
static void uart_driver_do_read(Context *ctx, GenMessage gen_message)
{
GlobalContext *glb = ctx->global;
struct UARTData *uart_data = ctx->platform_data;
term pid = gen_message.pid;
term ref = gen_message.ref;
uint64_t ref_ticks = term_to_ref_ticks(ref);
libs/estdlib/src/erlang.erl
Outdated
| -type raise_stacktrace() :: | ||
| [{module(), atom(), arity() | [term()]} | {function(), arity() | [term()]}] | stacktrace(). | ||
|
|
||
| -type monitor_option() :: {'alias', 'explicit_unalias' | 'demonitor' | 'reply_demonitor'}. |
There was a problem hiding this comment.
This is not enforced by style guide but usually we don't quote atoms when it's not required.
src/libAtomVM/context.c
Outdated
| } | ||
| // Prepare the message on ctx's heap which will be freed afterwards. | ||
| term ref = term_from_ref_ticks(monitored_monitor->ref_ticks, &ctx->heap); | ||
| term ref = term_make_process_reference(target->process_id, monitored_monitor->ref_ticks, &ctx->heap); |
There was a problem hiding this comment.
This is not what OTP does and there probably is a good reason.
Regular monitors are smaller than aliases.
7> Pid = spawn(fun() -> receive ok -> ok end end).
<0.94.0>
8> monitor(process, Pid).
#Ref<0.3008598808.1200095248.57512>
9> monitor(process, Pid, [{alias, explicit_unalias}]).
#Ref<0.0.11651.3008598808.1200160784.57533>
10> make_ref().
#Ref<0.3008598808.1200095248.57553>
src/libAtomVM/term.h
Outdated
| boxed_value[0] = TERM_BOXED_PROCESS_REF_HEADER; | ||
|
|
||
| #if TERM_BYTES == 4 | ||
| boxed_value[1] = (ref_ticks >> 4); |
There was a problem hiding this comment.
This was fixed for regular refs, we probably want ref_ticks >> 32
| end, | ||
| Term. | ||
|
|
||
| is_atomvm_or_otp_version_at_least(OTPVersion) -> |
There was a problem hiding this comment.
get_otp_version/0 already exists in this module.
Also there is a poor usage (fixed in #2025 )
OTPVersion = get_otp_version(),
if
OTPVersion =:= atomvm orelse OTPVersion >= 26 ->
B == erlang:term_to_binary(A);
true ->
true
end.
be sure to rather do:
OTPVersion = get_otp_version(),
if
OTPVersion >= 26 ->
B == erlang:term_to_binary(A);
true ->
true
end.
as atoms sort higher than integers.
There was a problem hiding this comment.
Also there is a poor usage
I would call it 'a readable way' :P
ce57282 to
eac7682
Compare
|
@pguyot the reason for the STM32 tests failing was that the size of a regular reference is actually 3 terms there, which means process reference takes 4 terms, and it probably conflicts with resource reference size, which is always 4 terms. I changed the process reference size to 5, but that's not a perfect solution :P So we need to come up with another way of distinguishing references - do you have anything in mind? I also added |
f759361 to
fe5d3c7
Compare
fe5d3c7 to
c882f24
Compare
src/libAtomVM/jit.c
Outdated
| } | ||
| } else if (term_is_local_pid_or_port(recipient_term)) { | ||
| int local_process_id; | ||
| if (term_is_local_pid_or_port(recipient_term)) { |
There was a problem hiding this comment.
This condition was checked on line 704.
| #define BOXED_INT64_SIZE (BOXED_TERMS_REQUIRED_FOR_INT64 + 1) | ||
| #define BOXED_FUN_SIZE 3 | ||
| #define FLOAT_SIZE (sizeof(float_term_t) / sizeof(term) + 1) | ||
| #define REF_SIZE ((int) ((sizeof(uint64_t) / sizeof(term)) + 1)) |
There was a problem hiding this comment.
A lot of downstream code uses REF_SIZE. If we want to rename it, we shoud keep the previous macro with a deprecation warning.
| #define TERM_BOXED_REFERENCE_RESOURCE_SIZE 5 | ||
| #endif | ||
| #define TERM_BOXED_REFERENCE_RESOURCE_HEADER (((TERM_BOXED_REFERENCE_RESOURCE_SIZE - 1) << 6) | TERM_BOXED_REF) | ||
| #define TERM_BOXED_RESOURCE_SIZE TERM_BOXED_REFERENCE_RESOURCE_SIZE |
There was a problem hiding this comment.
We can distinguish reference types by size (instead of a common header) and make resources larger than process aliases since aliases should be more commonly created. However, I would welcome static asserts to ensure sizes and headers are indeed different.
Also, we probably want the same prefix TERM_BOX_REFERENCE_*
In this current implementation, we have:
| type | 32 bits | 64 bits |
|---|---|---|
| short refs | 3 words | 2 words |
| process aliases | 4 words | 3 words |
| resources | 5 words (2 unused) | 4 words (1 unused) |
We could also have:
| type | 32 bits | 64 bits |
|---|---|---|
| short refs | 3 words | 2 words |
| process aliases | 4 words | 4 words |
| resources | 4 words | 4 words |
and distinguish process aliases from resources with last word being INVALID_PROCESS_ID for resources.
There was a problem hiding this comment.
I'd prefer sticking to the current option for now if it's acceptable, and change it in a separate PR
src/libAtomVM/term.h
Outdated
|
|
||
| typedef struct RefData RefData; | ||
|
|
||
| struct RefData |
There was a problem hiding this comment.
RefData is an excellent initiative, howerer it should be a way to serialize/deserialize any ref (local or external). And also we would need a TERM_BOXED_REFERENCE_MAX_SIZE to always be able to deserialize it.
| } | ||
| case CONTEXT_MONITOR_ALIAS: { | ||
| free(monitor); | ||
| } |
There was a problem hiding this comment.
We may want a break here line what was there on line 800.
src/libAtomVM/nifs.c
Outdated
| // msg is not in the port's heap | ||
| NativeHandlerResult result = NativeContinue; | ||
| if (UNLIKELY(memory_ensure_free_opt(ctx, 12, MEMORY_CAN_SHRINK) != MEMORY_GC_OK)) { | ||
| if (UNLIKELY(memory_ensure_free_opt(ctx, 13, MEMORY_CAN_SHRINK) != MEMORY_GC_OK)) { |
There was a problem hiding this comment.
12 was a mistake, we want a sum of constants.
src/libAtomVM/nifs.c
Outdated
| *is_alias = false; | ||
| while (term_is_nonempty_list(monitor_opts)) { | ||
| term option = term_get_list_head(monitor_opts); | ||
| if (term_is_tuple(option) && term_get_tuple_element(option, 0) == ALIAS_ATOM) { |
There was a problem hiding this comment.
We need to check the tuple size.
src/libAtomVM/nifs.c
Outdated
| default: | ||
| RAISE_ERROR(BADARG_ATOM); | ||
| } | ||
| } else if (term_is_tuple(option) && term_get_tuple_element(option, 0) == TAG_ATOM) { |
There was a problem hiding this comment.
We need to check the tuple size.
src/libAtomVM/term.h
Outdated
| // If you change a reference size, make sure it doesn't | ||
| // conflict with other reference sizes on all architectures. | ||
| #define SHORT_REF_SIZE ((int) ((sizeof(uint64_t) / sizeof(term)) + 1)) | ||
| #define TERM_BOXED_PROCESS_REF_SIZE SHORT_REF_SIZE + 1 |
There was a problem hiding this comment.
Macro value should be parenthesized (see AVMCCS-L016)
Signed-off-by: Mateusz Front <mateusz.front@swmansion.com>
Signed-off-by: Mateusz Front <mateusz.front@swmansion.com>
Signed-off-by: Mateusz Front <mateusz.front@swmansion.com>
|
Which clang-format version are you using? This causes some friction, but I'm not sure yet if just updating clang-format is the right thing to do. |
There was a problem hiding this comment.
This has to be changed:
/home/runner/work/AtomVM/AtomVM/src/libAtomVM/term.h:3148:12: error: ‘ref_data’ may be used uninitialized [-Werror=maybe-uninitialized]
3148 | return ref_data;
| ^~~~~~~~
/home/runner/work/AtomVM/AtomVM/src/libAtomVM/term.h: In function ‘process_console_mailbox’:
/home/runner/work/AtomVM/AtomVM/src/libAtomVM/term.h:3129:13: note: ‘ref_data’ declared here
3129 | RefData ref_data;
| ^~~~~~~~
In function ‘term_to_ref_data’,
inlined from ‘process_console_message’ at /home/runner/work/AtomVM/AtomVM/src/libAtomVM/nifs.c:1163:28,
inlined from ‘process_console_mailbox’ at /home/runner/work/AtomVM/AtomVM/src/libAtomVM/nifs.c:1229:18:
/home/runner/work/AtomVM/AtomVM/src/libAtomVM/term.h:3148:12: error: ‘ref_data’ may be used uninitialized [-Werror=maybe-uninitialized]
3148 | return ref_data;
| ^~~~~~~~
/home/runner/work/AtomVM/AtomVM/src/libAtomVM/term.h: In function ‘process_console_mailbox’:
/home/runner/work/AtomVM/AtomVM/src/libAtomVM/term.h:3129:13: note: ‘ref_data’ declared here
3129 | RefData ref_data;
d6ce43f to
ab5f3c8
Compare
|
@bettio I have |
| #define TERM_BOXED_REFERENCE_RESOURCE_SIZE 5 | ||
| #endif | ||
| #define TERM_BOXED_REFERENCE_RESOURCE_HEADER (((TERM_BOXED_REFERENCE_RESOURCE_SIZE - 1) << 6) | TERM_BOXED_REF) | ||
| #define TERM_BOXED_RESOURCE_SIZE TERM_BOXED_REFERENCE_RESOURCE_SIZE |
| #define BOXED_INT64_SIZE (BOXED_TERMS_REQUIRED_FOR_INT64 + 1) | ||
| #define BOXED_FUN_SIZE 3 | ||
| #define FLOAT_SIZE (sizeof(float_term_t) / sizeof(term) + 1) | ||
| #define REF_SIZE ((int) ((sizeof(uint64_t) / sizeof(term)) + 1)) |
| uint64_t ref_ticks = term_to_ref_ticks(t); | ||
|
|
||
| // Update also REF_AS_CSTRING_LEN when changing this format string | ||
| return fun->print(fun, "#Ref<%" PRId32 ".%" PRIu32 ".%" PRIu32 ">", process_id, (uint32_t) (ref_ticks >> 32), (uint32_t) ref_ticks); |
There was a problem hiding this comment.
Good catch! We should update tests as well to ensure comparison works. I added some tests when I made resources references.
There was a problem hiding this comment.
I'll add tests, but monitors are still uniquely identified by ref_ticks, so comparison should work as expected
There was a problem hiding this comment.
Ok, so it worked as I expected, but apparently not the way the BEAM does it. Should be fine now.
Signed-off-by: Mateusz Front <mateusz.front@swmansion.com>
Signed-off-by: Mateusz Front <mateusz.front@swmansion.com>
| uint32_t port_process_id; | ||
| uint32_t owner_process_id; | ||
| uint64_t ref_ticks; | ||
| RefData ref_data; |
There was a problem hiding this comment.
This comment applies to all the other port drivers here around.
In port.erl we have:
call(Port, Message, Timeout) ->
MonitorRef = monitor(port, Port),by default erlang:monitor/2 doesn't make aliases, and all the port calls are going through port:call so actually I don't think this change is really required.
TL;DR: I think this change adds some extra complexity for something it's not supposed to happen.
On the other side, supporting aliases in port drivers would require changing every 3rd party port driver as well (with no benefits in terms of performances)
@mat-hek let me know if there is anything wrong in my reasoning.
There was a problem hiding this comment.
You're right ;) These changes were needed because I changed all monitor refs to be process refs. Now I changed that back. The reason I kept RefData in drivers is that I assumed that we could still get a process ref there. If that's not the case, I'll revert it.
From my perspective, though, it's hard to figure out what kind of reference we can expect where, and keeping ref_ticks alone when ref_data is needed wouldn't fail immediately, but lead to hard-to-debug issues.
Signed-off-by: Mateusz Front <mateusz.front@swmansion.com>
Signed-off-by: Mateusz Front <mateusz.front@swmansion.com>
Signed-off-by: Mateusz Front <mateusz.front@swmansion.com>
Signed-off-by: Mateusz Front <mateusz.front@swmansion.com>
bettio
left a comment
There was a problem hiding this comment.
Review is still in progress, but meanwhile I submit comments for what I reviewed.
| %% @end | ||
| %%----------------------------------------------------------------------------- | ||
| -spec monitor | ||
| (Type :: process, Pid :: pid() | atom(), [monitor_option()]) -> reference(); |
There was a problem hiding this comment.
monitor can be also used with atoms or with tuples for processes on remote nodes.
I suggest introducing monitor_process_identifier() as Erlang does.
There was a problem hiding this comment.
This is copy-pasted from the typespec above, I think it should be fixed in a separate PR
| } | ||
| globalcontext_get_process_unlock(glb, p); | ||
| } | ||
| } else if (!term_is_reference(recipient_term)) { |
There was a problem hiding this comment.
Personally I find more readable having just } else { at the end, instead of the previous condition negated.
There was a problem hiding this comment.
This is not the previous condition negated
| TRACE_SEND(ctx, x_regs[0], x_regs[1]); | ||
| globalcontext_send_message(ctx->global, local_process_id, x_regs[1]); | ||
| } else if (term_is_process_reference(recipient_term)) { | ||
| int32_t local_process_id = term_process_ref_to_process_id(recipient_term); |
There was a problem hiding this comment.
This same code block is duplicated in jit.c. I suggest creating a helper function to avoid code duplication, such as: globalcontext_send_message_to_alias(GlobalContext *, term alias, term message);
There was a problem hiding this comment.
Actually, the whole jit_send function is a duplicate of the case OP_SEND in opcodesswitch, so I believe it all should be unified.
| globalcontext_send_message(ctx->global, local_process_id, ctx->x[1]); | ||
| ctx->x[0] = ctx->x[1]; | ||
| } else if (term_is_process_reference(recipient_term)) { | ||
| int32_t process_id = term_process_ref_to_process_id(recipient_term); |
There was a problem hiding this comment.
See my comment about duplicated code in opcodesswitch.h.
| check(Sorted, [A, B, C, D, E]) + | ||
| bool_to_n(Sorted < [make_ref()]) * 2 + | ||
| erlang:display([A, C, E, B, D]), | ||
| erlang:display(Sorted), |
There was a problem hiding this comment.
I think erlang:display() here is a leftover.
src/libAtomVM/term.h
Outdated
| return ret; | ||
| } | ||
|
|
||
| static inline RefData term_to_ref_data(term t) |
There was a problem hiding this comment.
As soon as the last port change is reverted, I think this will be an orphan function. If so, I suggest removing it.
(I might be wrong since I'm still doing the review)
Actually as an additional feedback, I'm not sure this struct return is a good pattern, usually is more common in C using pointers when passing around structs.
See also AVMCCS-A007 in C_CODING_STYLE.md guide.
| const uint32_t *words; | ||
| }; | ||
|
|
||
| typedef struct RefData RefData; |
There was a problem hiding this comment.
See my comment about term_to_ref_data(term t).
There was a problem hiding this comment.
I've seen, how does it affect this line?
| RefData ref_data = { | ||
| .type = RefTypeProcess, | ||
| .process = { .ref_ticks = globalcontext_get_ref_ticks(ctx->global), .process_id = ctx->process_id } | ||
| }; |
There was a problem hiding this comment.
As a feedback I suggest a shorter lines here:
RefData ref_data = {
.type = RefTypeProcess,
.process = {
.ref_ticks = globalcontext_get_ref_ticks(ctx->global),
.process_id = ctx->process_id
}
};There was a problem hiding this comment.
Maybe a line-length limit in clang-format would be a good idea?
src/libAtomVM/term.h
Outdated
| return ref_data; | ||
| } | ||
|
|
||
| static inline term term_from_ref_data(RefData ref_data, Heap *heap) |
There was a problem hiding this comment.
See AVMCCS-A007 in C_CODING_STYLE.md guide.
However my question is: after reverting all the changes to port drivers, are we just using term_from_ref_data for process references? If so, this function might be way more complex than what we need now in our codebase now.
There was a problem hiding this comment.
RefData can store any ref, as requested by @pguyot here: #2027 (comment). Please have a look at @pguyot's review, as I have an impression your preferences differ on this matter
dc30e6b to
a0f7887
Compare
TODO:
spawn_optThese changes are made under both the "Apache 2.0" and the "GNU Lesser General
Public License 2.1 or later" license terms (dual license).
SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later