Getting started guide fails?

[Lorquas]

I'm running through the getting started guide, found in this repo's README.md and off your vimeo page. I'm going along and everything is peachy until:

❯❯❯ snapctl task create -t mock-file.yaml
Using task manifest to create task
Error creating task:
Plugin not found: type(publisher) name(mock-file) version(-1)

I open up the yaml and check out that its looking for a publisher named "mock-file" and not "file", run snapctl plugin unload publisher:file:1 followed by
snapctl plugin load plugins/snap-plugin-publisher-mock-file . I then run the task create again:

❯❯❯ snapctl task create -t mock-file.yaml
Using task manifest to create task
Task created
ID: cd9697b4-dcc4-4e3b-a8c2-1d4955bae9be
Name: Task-cd9697b4-dcc4-4e3b-a8c2-1d4955bae9be

seems better, but now:

❯❯❯ snapctl task list
ID                   NAME                        STATE       HIT     MISS    FAIL    CREATED         LAST FAILURE
cd9697b4-dcc4-4e3b-a8c2-1d4955bae9be     Task-cd9697b4-dcc4-4e3b-a8c2-1d4955bae9be   Disabled    10      0   10      12:01PM 8-01-2016   Publish call error: EOF

and in the console output of snapd:

DEBU[2016-08-01T12:01:14-04:00] level=debug msg="publishing started"          _module=plugin-exec io=stderr plugin=snap-plugin-publisher-mock-file
DEBU[2016-08-01T12:01:14-04:00] level=error msg="error decoding" content=[13 255 147 2 1 2 255 148 0 1 255 136 0 0 255 155 255 135 3 1 1 10 77 101 116 114 105 99 84 121 112 101 1 255 136 0 1 9 1 10 78 97 109 101 115 112 97 99 101 95 1 255 140 0 1 19 76 97 115 116 65 100 118 101 114 116 105 115 101 100 84 105 109 101 95 1 255 142 0 1 8 86 101 114 115 105 111 110 95 1 4 0 1 7 67 111 110 102 105 103 95 1 255 144 0 1 5 68 97 116 97 95 1 16 0 1 5 84 97 103 115 95 1 255 146 0 1 5 85 110 105 116 95 1 12 0 1 12 68 101 115 99 114 105 112 116 105 111 110 95 1 12 0 1 10 84 105 109 101 115 116 97 109 112 95 1 255 142 0 0 0 38 255 139 2 1 1 23 91 93 99 111 114 101 46 78 97 109 101 115 112 97 99 101 69 108 101 109 101 110 116 1 255 140 0 1 255 138 0 0 65 255 137 3 1 1 16 78 97 109 101 115 112 97 99 101 69 108 101 109 101 110 116 1 255 138 0 1 3 1 5 86 97 108 117 101 1 12 0 1 11 68 101 115 99 114 105 112 116 105 111 110 1 12 0 1 4 78 97 109 101 1 12 0 0 0 16 255 141 5 1 1 4 84 105 109 101 1 255 142 0 0 0 10 255 143 5 1 2 255 150 0 0 0 33 255 145 4 1 1 17 109 97 112 91 115 116 114 105 110 103 93 115 116 114 105 110 103 1 255 146 0 1 12 1 12 0 0] error=EOF   _module=plugin-exec io=stderr plugin=snap-plugin-publisher-mock-file
ERRO[2016-08-01T12:01:14-04:00] error with publisher job                      _module=scheduler-job block=run content-type=snap.gob error=Publish call error: EOF job-type=publisher plugin-config=map[file:{Value:/tmp/snap_published_mock_file.log} debug:{Value:true}] plugin-name=mock-file plugin-version=-1
WARN[2016-08-01T12:01:14-04:00] Publish job failed 

Until the fail limit of 10 consecutive fails is reached and it aborts completely.

I've retried the whole process using the plain "file" plugin (and rewriting the yaml), I've also changed the target file to a file/folder with 777 permissions, selinux and firewalls off. And I get the same thing every time. Being as new to this framework as I am (following the getting started guide), any ideas as to what could be wrong?

[IzabellaRaulin]

Hello @Lorquas, could you start from the beginning and do the following steps once again as below:
In one terminal window:

snapd -l 1 -t 0

In another terminal window in snap directory:

snapctl plugin load build/plugin/snap-plugin-collector-mock1
snapctl plugin load build/plugin/snap-plugin-processor-passthru
snapctl plugin load build/plugin/snap-plugin-publisher-mock-file

snapctl task create -t examples/tasks/mock-file.yaml

When I execute these commands above I don't receive any error:

snapctl task list
ID                                       NAME                                            STATE           HIT     MISS    FAIL    CREATED                 LAST FAILURE
986a5089-06c4-40c8-81df-13834e959372     Task-986a5089-06c4-40c8-81df-13834e959372       Running         11      0       0       6:21PM 8-01-2016

However, your issue might point that there is some bug which occures under some circumstances.

[Lorquas]

Here is the full console logs of running this:
snapd.txt
snapctl.txt

Note that I am using both snap and plugins from the latest release here: https://github.com/intelsdi-x/snap/releases/tag/v0.15.0-beta and not from the git repo directly.

[thomastaylor312]

@Lorquas This could be because we renamed the included file publishing plugin in this PR. The current getting started guide is using v0.13.0 which was released before the aforementioned PR was merged. With v0.15.0-beta there is a snap-plugin-publisher-mock-file that you should be able to load as @IzabellaRaulin mentioned. Can you try it and let us know if there is still an issue?

As for the documentation, @mjbrender is there already any ongoing tasks to update the documentation or should I create a new issue?

[Lorquas]

@thomastaylor312 yup, I've tried both the snap-plugin-publisher-file and snap-plugin-publisher-mock-file. The former won't even submit the task due to a mismatch in plugin names (this makes sense with the rename). The snap-plugin-publisher-mock-file plugin gets the EOF file errors as seen in my last comment.

[ConnorDoyle]

Duplicating selected lines from snapd logs for convenience:

DEBU[2016-08-01T13:14:16-04:00] Submitting publish job                        _block=submit-publish-job _module=scheduler-workflow parent-node-type=processor publish-name=mock-file publish-version=-1 task-id=52523658-c74d-409c-b3b0-ad974e1f29ca task-name=Task-52523658-c74d-409c-b3b0-ad974e1f29ca
DEBU[2016-08-01T13:14:16-04:00] starting publisher job                        _module=scheduler-job block=run content-type=snap.gob job-type=publisher plugin-config=map[debug:{Value:true} file:{Value:/tmp/snap_published_mock_file.log}] plugin-name=mock-file plugin-version=-1
DEBU[2016-08-01T13:14:16-04:00] plugin selected                               _module=control-routing block=select hitcount=0 index=publisher:mock-file:v3:id1 pool size=1 strategy=least-recently-used
DEBU[2016-08-01T13:14:16-04:00] level=debug msg="publishing started"          _module=plugin-exec io=stderr plugin=snap-plugin-publisher-mock-file
DEBU[2016-08-01T13:14:16-04:00] level=error msg="error decoding" content=[13 255 147 2 1 2 255 148 0 1 255 136 0 0 255 155 255 135 3 1 1 10 77 101 116 114 105 99 84 121 112 101 1 255 136 0 1 9 1 10 78 97 109 101 115 112 97 99 101 95 1 255 140 0 1 19 76 97 115 116 65 100 118 101 114 116 105 115 101 100 84 105 109 101 95 1 255 142 0 1 8 86 101 114 115 105 111 110 95 1 4 0 1 7 67 111 110 102 105 103 95 1 255 144 0 1 5 68 97 116 97 95 1 16 0 1 5 84 97 103 115 95 1 255 146 0 1 5 85 110 105 116 95 1 12 0 1 12 68 101 115 99 114 105 112 116 105 111 110 95 1 12 0 1 10 84 105 109 101 115 116 97 109 112 95 1 255 142 0 0 0 38 255 139 2 1 1 23 91 93 99 111 114 101 46 78 97 109 101 115 112 97 99 101 69 108 101 109 101 110 116 1 255 140 0 1 255 138 0 0 65 255 137 3 1 1 16 78 97 109 101 115 112 97 99 101 69 108 101 109 101 110 116 1 255 138 0 1 3 1 5 86 97 108 117 101 1 12 0 1 11 68 101 115 99 114 105 112 116 105 111 110 1 12 0 1 4 78 97 109 101 1 12 0 0 0 16 255 141 5 1 1 4 84 105 109 101 1 255 142 0 0 0 10 255 143 5 1 2 255 150 0 0 0 33 255 145 4 1 1 17 109 97 112 91 115 116 114 105 110 103 93 115 116 114 105 110 103 1 255 146 0 1 12 1 12 0 0] error=EOF   _module=plugin-exec io=stderr plugin=snap-plugin-publisher-mock-file
ERRO[2016-08-01T13:14:16-04:00] error with publisher job                      _module=scheduler-job block=run content-type=snap.gob error=Publish call error: EOF job-type=publisher plugin-config=map[debug:{Value:true} file:{Value:/tmp/snap_published_mock_file.log}] plugin-name=mock-file plugin-version=-1
WARN[2016-08-01T13:14:16-04:00] Publish job failed                            _block=submit-publish-job _module=scheduler-workflow parent-node-type=processor publish-name=mock-file publish-version=-1 task-id=52523658-c74d-409c-b3b0-ad974e1f29ca task-name=Task-52523658-c74d-409c-b3b0-ad974e1f29ca

[jcooklin]

@Lorquas I'm trying to reproduce the error as you are seeing it.

• Are you on Linux or Mac>
• Did you download the v0.15 release of both snap and the plugins? (or did you perhaps compile snapd yourself?)

[Lorquas]

@jcooklin I'm running on Linux. Doing more testing it seems to not work on Fedora 23 or 24, but does seem to work on an older RHEL 7.2 release.

I'm downloading and running the binaries/plugins from: snap-plugins-v0.15.0-beta-linux-amd64.tar.gz and snap-v0.15.0-beta-linux-amd64.tar.gz

[jcooklin]

@Lorquas A PR has been submitted to fix this issue. You can download the binaries built with the fix here. Will you test the fix in your environment? (I was able to reproduce the error you reported on Ubuntu 16.04 but not on 14.04 or 12.04.)

The issue was due to the fact that the last advertised time field was not getting set on the metrics returned to the framework by the mock1 plugin and on newer OS's when the default (empty) value came through it had a fractional time zone offset which, as it turns out, gob serialization does not like (Time.MarshalBinary: zone offset has fractional minute).

[lmroz]

@jcooklin this is strange, I'd avoid resolving this by setting metric time to Now() especially during conversion. If time.Time is empty it's .Location() should be &UTC https://golang.org/src/time/time.go?s=23385:24735#L810, and marshalling should recognize that. Perhaps it was set previously using time.Unix() and when marshalled again but not using grpc but gob it crashes.

[Lorquas]

@jcooklin I've tested your build on all of my systems and things are working. I think this issue can be resolved with your PR and the documentation tweaks as @thomastaylor312 suggested.

Thanks everyone for the quick replies!