From d098cf66624230390d762e9995e8fd61b0d3c71d Mon Sep 17 00:00:00 2001 From: Mart Lubbers Date: Sun, 2 Jul 2017 13:51:26 +0200 Subject: [PATCH] elaborate on share usage --- arch.communication.tex | 5 +- arch.devices.tex | 4 +- arch.itasks.tex | 220 ++++++++++++++++++++++---------------- conclusion.discussion.tex | 10 ++ 4 files changed, 140 insertions(+), 99 deletions(-) diff --git a/arch.communication.tex b/arch.communication.tex index 7da8ad1..18bc076 100644 --- a/arch.communication.tex +++ b/arch.communication.tex @@ -2,8 +2,8 @@ The communication from the server to the client and vice versa is just a character stream containing encoded \gls{mTask} messages. The \CI{synFun} belonging to the device is responsible for sending the content in the left channel and putting received messages in the right channel. Moreover, the -boolean value should be set to \CI{True} when the connection is terminated. The -specific encoding of the messages is visible in +boolean flag in the channel type should be set to \CI{True} when the connection +is terminated. The specific encoding of the messages is visible in Appendix~\ref{app:communication-protocol}. The type holding the messages is shown in Listing~\ref{lst:avmsg}. Detailed explanation about the message types and according actions will be given in the following subsections. @@ -51,7 +51,6 @@ brevity. } \end{lstlisting} - \subsection{Add a device} A device can be added by filling in the \CI{MTaskDevice} record as much as possible and running the \CI{connectDevice} function. This function grabs the diff --git a/arch.devices.tex b/arch.devices.tex index b43a40f..db56b48 100644 --- a/arch.devices.tex +++ b/arch.devices.tex @@ -182,12 +182,12 @@ void run_task(struct task *t){ pc+=2; break; // ... - case BCADD: trace("add"); + case BCADD: stack[sp-2] = stack[sp-2] + stack[sp-1]; sp -= 1; break; // ... - case BCJMPT: trace("jmpt to %d", program[pc]); + case BCJMPT: pc = stack[--sp] ? program[pc]-1 : pc+1; break; // ... diff --git a/arch.itasks.tex b/arch.itasks.tex index 7a21471..040b287 100644 --- a/arch.itasks.tex +++ b/arch.itasks.tex @@ -25,16 +25,16 @@ the reference to the device and not the actual device record. | SerialDevice TTYSettings | ... :: MTaskDevice = - { deviceTask :: Maybe TaskId - , deviceError :: Maybe String - , deviceChannels :: String - , deviceName :: String - , deviceState :: BCState - , deviceTasks :: [MTaskTask] - , deviceResource :: MTaskResource - , deviceSpec :: Maybe MTaskDeviceSpec - , deviceShares :: [MTaskShare] - } + { deviceTask :: Maybe TaskId + , deviceError :: Maybe String + , deviceChannels :: String + , deviceName :: String + , deviceState :: BCState + , deviceTasks :: [MTaskTask] + , deviceResource :: MTaskResource + , deviceSpec :: Maybe MTaskDeviceSpec + , deviceShares :: [MTaskShare] + } channels :: MTaskDevice -> Shared Channels @@ -44,19 +44,23 @@ class MTaskDuplex a where The \CI{deviceResource} component of the record must implement the \CI{MTaskDuplex} interface that provides a function that launches a \gls{Task} -used for synchronizing the channels. The \CI{deviceTask} stores the -\gls{Task}-id for this \gls{Task} when active so that it can be checked upon. -This top-level task has the duty to report exceptions and errors as they are -thrown by setting the \CI{deviceError} field. All communication goes via these -channels. To send a message to the device, the system just puts it -in the channels. Messages sent from the client to the server are also placed -in there. In the case of the \gls{TCP} device type, the \gls{Task} is just a -simple wrapper around the existing \CI{tcpconnect} function in \gls{iTasks}. In -case of a device connected by a serial connection, it uses the newly developed -serial port library of \gls{Clean}\footnote{\url{% -https://gitlab.science.ru.nl/mlubbers/CleanSerial}}. -The implementation and semantics for the \CI{MTaskMSGRecv} and -\CI{MTaskMSGSend} types is given in Section~\ref{sec:communication}. +used for synchronizing the channels. The \CI{deviceChannels} field can be used +to get the memory \gls{SDS} containing the channels. This field does not +contain the channels itself because they update often. The field is used to +get a memory \gls{SDS} containing the actual channel data when calling the +\CI{channels} function. The \CI{deviceTask} stores the \gls{Task}-id for this +\gls{Task} when active so that it can be checked upon. This top-level task has +the duty to report exceptions and errors as they are thrown by setting the +\CI{deviceError} field. All communication goes via these channels. To send a +message to the device, the system just puts it in the channels. Messages sent +from the client to the server are also placed in there. In the case of the +\gls{TCP} device type, the \gls{Task} is just a simple wrapper around the +existing \CI{tcpconnect} function in \gls{iTasks}. In case of a device +connected by a serial connection, it uses the newly developed serial port +library of \gls{Clean}\footnote{\url{% +https://gitlab.science.ru.nl/mlubbers/CleanSerial}}. The implementation and +semantics for the \CI{MTaskMSGRecv} and \CI{MTaskMSGSend} types are given in +Section~\ref{sec:communication}. Besides all the communication information, the record also keeps track of the \glspl{Task} currently on the device, the compiler state (see @@ -66,67 +70,97 @@ is given in Listing~\ref{lst:mtaskdevice}. The definitions of the message format are explained in the following section. \subsection{Shares} -\Glspl{SDS} on the device are synchronized in the server and thus stored both -on the server and on the client. In this way the communication is kept to the -bare minimum because no communication with the device is needed for a -\gls{SDS} read operation. In this case the \gls{Task} requesting can just be -provided with the synchronized value. As mentioned before, the device has to -explicitly publish an update. This has the implication that the server and the -client can get out of sync. However, this is not a problem since it is well -documented and by design. +\Glspl{SDS} on the device can be accessed by both the device and the server. +While it would be possible to only store the \glspl{SDS} on the device, this +would require a lot of communication because every read operation will then +result in sending messages to-and-fro the device. Thus, the \gls{Task} +requesting the shared information can just be provided with the synchronized +value. As mentioned before, the device has to explicitly publish an update. +This has the implication that the server and the client can get out of sync. +However, this is by design and well documented. In the current system, an +\gls{SDS} can only reside on a single device. % TODO -The system keeps track of the \glspl{SDS} stored on the client in a big -\gls{SDS} containing a list of devices. Client-\glspl{SDS} can be stored on one -device at the same time. This means that if an \gls{SDS} updates, everyone -watching it will be notified. This would result in a lot of notifications that -are not meant for the watcher. Moreover, when a client updates the \gls{SDS} -this is processed by the connection handler and results in an update of the -real \gls{SDS}. Finally, the \gls{SDS} of a client must be synchronized with -the actual device. Thus, when an \gls{iTasks}-\gls{Task} writes the -client-\gls{SDS}, it must be propagated to the real device. There are several -ways of tackling this problem each with their own level of granularity. - -First an actual \gls{iTasks}-\gls{SDS} for every \gls{SDS} used in a client can -be instantiated with one \gls{iTasks}-\gls{Task} listening to the \gls{SDS} and -synchronizing it with the device when an update occurred. This approach is very -expensive as it requires a lot of listening \glspl{Task}. - -Improved on this, a single \gls{iTasks}-\gls{SDS} can be created for every -devices that stores the respective \glspl{SDS}. Using the \CI{mapReadWrite} -functions, a single \gls{SDS} per device can be created as a lens that allows -mapping on a single client-\gls{SDS}. However, this approach still requires -\glspl{Task} listening to the \gls{SDS} and when an \gls{SDS} is written, -everyone is notified, even if the \gls{Task} only uses the value of a single -different \gls{SDS}. - -Finally, the current approach --- a single \gls{SDS} for the entire system ---- was explored. To create \glspl{SDS} per device or per client-\glspl{SDS} a -\CI{mapReadWrite} can be used but it suffers from the same problem as mentioned -before. Moreover, a \gls{Task} still has to watch the \gls{SDS} and communicate -the client-\gls{SDS} updates to the actual device. Both of these problems can -be solved by using a tailor made \gls{SDS} that heavily depends on parametric -lenses. +% Single share per share +There are several possible approaches for storing \glspl{SDS} on the server +each with their own level of control. A possible way is to --- in the device +record --- add a list of references to \gls{iTasks}-\glspl{SDS} that represent +the \gls{SDS} on the device. The problem with this is the fact that an +\gls{SDS} can become an orphan. The \gls{SDS} is still accessible even when the +device is long gone. There is no way of knowing whether the \gls{SDS} is +unreachable because of the device being gone, or the \gls{SDS} itself is gone +on the device. Accessing the \gls{SDS} happens by calling the \CI{get}, +\CI{set} and \CI{upd} functions directory on the actual \gls{SDS}. + +% Single share per device +Another approach would be to have reference to a \gls{SDS} containing a table +of \gls{SDS} values per device. This approach poses the same orphan problem as +before. Accessing a single \gls{SDS} on the device happens by calling the +\CI{get}, \CI{set} and \CI{upd} functions on the actual table \gls{SDS} with an +applied \CI{mapReadWrite}. Using parametric lenses can circumvent problem of +watchers getting notified for other shares that are written. Error handling is +better than the previously mentioned approach because a \gls{SDS} can know +whether the \gls{SDS} has really gone because it will not be available anymore +in the table. It still does not know whether the device is still available. + +% One share to rule them all +Finally, all devices containing all of their \glspl{SDS} in a table could be +stored in a single big \gls{SDS}. While the \CI{mapReadWrite} functions require +a bit more logic, they can determine the source of the error and act upon it. +Also, the parametric lenses must contain more logic. A downside of this +approach is that updating a single \gls{SDS} requires an update of the entire +object. In practise, this is not a real issue since almost all information can +be reused and \gls{SDS} residing on devices are often not updated with a very +high frequency. \subsection{Parametric Lenses} -The type for the parametric lens of the big \gls{SDS} is \CI{Maybe -(MTaskDevice, Int)}. The \gls{SDS} is responsible for storing the entire list -of devices, from now on global. Moreover, the \gls{SDS} can focus on a single -device, from now on local. A local \gls{SDS} can also specifically focus on a -single \gls{SDS} on a single device, from now on called local-share. The -implementation of the real \gls{SDS} is given in Listing~\ref{lst:actualdev}. -The \gls{SDS} is a lens on an actual \gls{SDS} that writes to a file or memory. -Reading the \gls{SDS} is nothing more than reading the real \gls{SDS}. Writing -the \gls{SDS} is a little bit more involved. If the write operation originated -from an \gls{SDS} focussed on a single client-\gls{SDS}, the write action must -also be relayed to the actual device. If the write originated from an \gls{SDS} -focussed the devices or on one device only, nothing needs to be done. The -notification predicate determines whether a watcher gets a notification update. +The type for the parametric lens of the \gls{SDS} containing all devices is +\CI{Maybe (MTaskDevice, Int)}. There are several levels of abstraction that +have to be introduced. First, the \gls{SDS} responsible for storing the entire +list of devices is called the global \gls{SDS}. Secondly, a \gls{SDS} can focus +on a single device, such \glspl{SDS} are called local \glspl{SDS}. Finally, a +\gls{SDS} can focus on a single \gls{SDS} on a single device. These \glspl{SDS} +are called share \glspl{SDS}. Using parametric lenses, the notifications can be +directed to only the watchers interested. Moreover, using parametric lenses, +the \gls{SDS} can know whether it is updating a single \gls{SDS} on a single +device and synchronize the value with the actual device. This means that when +writing to a share \gls{SDS} the update is also transformed to messages that +are put in the channels of the corresponding device to also notify the device +of the update. The \gls{SDS} is tailor-made and uses an actual standard +\gls{SDS} that writes to a file or memory as the storage. The tailor-made read +and write functions are only used to detect whether it is required to send an +update to the actual device. + +Listing~\ref{lst:actualdev} shows the implementation of the big \gls{SDS}. From +this \gls{SDS} all other \glspl{SDS} are derived. The following paragraphs show +how this is achieved for the global \gls{SDS} local \gls{SDS} and the share +\gls{SDS}. In the big \gls{SDS}, reading the value is just a matter of reading +the standard \gls{SDS} that serves as the actual storage of the \gls{SDS}. The +derived shares will filter the output read accordingly. Writing the share +requires some extra work because it might be possible that an actual device has +to be notified. First, the actual storage of the \gls{SDS} is written. If the +parameter was \CI{Nothing} --- the global \gls{SDS} --- the write operation is +done. If the parameter was \CI{Just (d, -1)} --- a local \gls{SDS} --- nothing +has to be done as well. The final case is the special case, when the parameter +is \CI{Just (d, i)}, this means that the \gls{SDS} was focussed on device +\CI{d} and \gls{SDS} \CI{i} and thus it needs to write to it. First it locates +the device in the list, followed by the location of the share to check whether +is still exists. Finally the actual update messages are added to the device +channels using the \CI{sendMessagesIW} function. + +All of the methods share the same \CI{SDSNotifyPred p} which is a function +\CI{p -> Bool} and determines for the given \CI{p} whether a notification is +required. The predicate function has the \CI{p} of the writer curried in and +can determine whether the second argument --- the reader --- needs to be +notified. In practice, the reader only needs to be notified when the parameters +are exactly the same. \begin{lstlisting}[label={lst:actualdev},% caption={Device \gls{SDS}}] +($<) a fb = fmap (const a) fb + deviceStore :: RWShared (Maybe (MTaskDevice, Int)) [MTaskDevice] [MTaskDevice] -deviceStore = SDSSource {SDSSource | name="deviceStore", read = realRead, write= realWrite} +deviceStore = SDSSource {SDSSource | name="deviceStore", read=realRead, write=realWrite} where realRead :: (Maybe (MTaskDevice,Int)) *IWorld -> (MaybeError TaskException [MTaskDevice], *IWorld) realRead p iw = read realDeviceStore iw @@ -134,25 +168,16 @@ where realWrite :: (Maybe (MTaskDevice,Int)) [MTaskDevice] *IWorld -> (MaybeError TaskException (SDSNotifyPred (Maybe (MTaskDevice,Int))), *IWorld) realWrite mi w iw # (merr, iw) = write w realDeviceStore iw - | isError merr || isNothing mi = (merr $> notifyPred mi, iw) + | isError merr || isNothing mi = (merr $> gEq{|*|} mi, iw) # (Just (dev, ident)) = mi - | ident == -1 = (merr $> notifyPred mi, iw) + | ident == -1 = (merr $> gEq{|*|} mi, iw) = case find ((==)dev) w of Nothing = (Error $ exception "Device lost", iw) Just {deviceShares} = case find (\d->d.identifier == ident) deviceShares of Nothing = (Error $ exception "Share lost", iw) Just s = case sendMessagesIW [MTUpd ident s.MTaskShare.value] dev iw of (Error e, iw) = (Error e, iw) - (Ok _, iw) = (Ok $ notifyPred mi, iw) - - notifyPred :: (Maybe (MTaskDevice, Int)) (Maybe (MTaskDevice, Int)) -> Bool - notifyPred Nothing Nothing = True // Global watcher looking at a global event - notifyPred Nothing (Just _) = False // Global watcher looking at a local event - notifyPred (Just _) Nothing = False // Local watcher looking at a global event - // Local device watcher looking at a local event - notifyPred (Just (d1, -1)) (Just (d2, _)) = d1 == d2 - // Local share watcher looking at a local share event - notifyPred (Just (d1, i1)) (Just (d2, i2)) = d1 == d2 && i1 == i2 + (Ok _, iw) = (Ok $ gEq{|*|} mi, iw) realDeviceStore :: Shared [MTaskDevice] realDeviceStore = sharedStore "mTaskDevices" [] @@ -160,7 +185,9 @@ where \subsubsection{Global \glspl{SDS}} Accessing the global \gls{SDS} is just a matter of focussing the -\CI{deviceStore} with the \CI{Nothing} parameter as follows: +\CI{deviceStore} to \CI{Nothing}. In this way, \glspl{Task} watching the +\gls{SDS} will only be notified if a device is added or removed. The actual +code is as follows: \begin{lstlisting}[caption={Global \gls{SDS}}] deviceStoreNP :: Shared [MTaskDevice] @@ -168,10 +195,12 @@ deviceStoreNP = sdsFocus Nothing deviceStore \end{lstlisting} \subsubsection{Local \glspl{SDS}} -Accessing a single device can be done using the \CI{mapReadWrite} function. +Accessing a single device can be done using the \CI{deviceShare} function. Since device comparison is shallow, the device that is given is allowed to be an old version. The identification of devices is solely done on the name of the -channels and is unique throughout the system. The implementation is as follows: +channels and is unique throughout the system. This type of \gls{SDS} will only +be notified if the device itself changed. It will not be notified when only a +single \gls{SDS} on the device changes. The implementation is as follows: \begin{lstlisting}[caption={Local \gls{SDS}}] deviceShare :: MTaskDevice -> Shared MTaskDevice @@ -187,9 +216,12 @@ deviceShare d = mapReadWriteError \subsubsection{Local-\gls{SDS} specific \glspl{SDS}} A single \gls{SDS} on a single device can be accessed using the \CI{shareShare} -function. This function focusses the real big \gls{SDS} on a single \gls{SDS} -and uses the \CI{mapReadWrite} functions to serve the correct part of the -information. +function. This function focusses the global \gls{SDS} on a single \gls{SDS} +from a single device. It can use old share references in the same fashion as +the local \gls{SDS} only treating it as references. It uses the +\CI{mapReadWrite} functions to serve the correct part of the information. When +a \gls{Task} writes to this \gls{SDS}, the global \gls{SDS} will know this +through the parameter and propagate the value to the device. \begin{lstlisting}[caption={Local \gls{SDS}}] shareShare :: MTaskDevice MTaskShare -> Shared BCValue diff --git a/conclusion.discussion.tex b/conclusion.discussion.tex index 3b1657f..2bd77d3 100644 --- a/conclusion.discussion.tex +++ b/conclusion.discussion.tex @@ -91,6 +91,16 @@ use runtime typing with \CI{Dynamic}s or the encoding technique currently used for \CI{BCValue}s. Using \glspl{SDS} for multiple \glspl{Task} within one device is solved when the previous point is implemented. +Another way of improving on \gls{SDS} handling is to separate \glspl{SDS} from +devices. In this implementation, the \gls{SDS} not only needs to know on which +device it is, but also which internal device \gls{SDS} id it has. A pro of +this technique is that the \gls{SDS} can be shared between \glspl{Task} that +are not defined in the same scope because they are separated. A con of this +implementation is that the mechanisms for implementing \glspl{SDS} have to be +more complex, they have to keep track of the devices containing or sharing an +\gls{SDS}. Moreover, when the \gls{SDS} is updated, all attached devices must +be updated which requires some extra work. + \subsection{Robustness} \paragraph{Reconnect with lost devices:} The robustness of the system can be greatly improved. Devices that lose -- 2.20.1