Using Delphi to establish communication and data exchange servers - Analysis of Transceiver technology (Part 2) Author: 火鸟 [email protected] 2. Detailed explanation of Transceiver Service 1. Transceiver Service Analysis Summary Transceiver Service is the core component of the Transceiver system. Transceiver Kernel is responsible for reading the Port and Channel definitions and parameters set by the Transceiver Console from the system configuration library, dynamically creating and controlling communication ports and their relationships during runtime, and controlling data. Scheduling of receiving, sending and buffering, managing logs and queues, etc. Transceiver Shell is the implementation of all types of ports supported for data sending and receiving. 2. Transceiver Service Design Summary Transceiver Service is developed from Service application in Delphi. Service Application can run in system mode instead of user mode. The operating system Service Control Manager (SCM) is responsible for the operation management of the program. Service has no user interface and belongs to the system. background program. Transceiver Kernel is a series of methods of the Transceiver class that establishes and controls Transceiver Shell, and Transceiver Shell is a collection of objects responsible for communication. Note: Due to performance and load considerations, Transceiver Kernel only logically implements the functional division in the architecture diagram, and the constituent modules are not implemented in a completely object-based manner. 3． Transceiver Service Implementation Summary i. Create a Service Application. Select NEW|Other... from the Delphi main menu File... and select NEW|Service Application in the pop-up New Items dialog box. You can see that the generated program framework is as follows: PRogram Project1; uses SvcMgr, Unit1 in 'Unit1.pas' {Service1: TService};{$R *.RES}begin Application.Initialize; Application.CreateForm(TService1, Service1); Application.Run;end.unit Unit1;interfaceuses Windows, Messages, SysUtils, Classes, Graphics, Controls, SvcMgr, Dialogs;type TService1 = class(TService) private { Private declarations } public function GetServiceController: TServiceController; override; { Public declarations } end;var Service1: TService1;implementation{$R *.DFM}procedure ServiceController(CtrlCode: DWord); stdcall;begin Service1.Controller(CtrlCode);end;function TService1.GetServiceController: TServiceController;begin Result := ServiceController;end;end. You can see that in addition to the SvcMgr used for service management referenced in the uses unit, TService1 inherits from TServiced instead of TForm, an overloaded GetServiceController function and the ServiceController process called in stdcall mode, it is created with Delphi There is not much special about a service program, Delphi Fans may want to cheer again, this is the powerful charm of Delphi RAD. In addition, since the Service Application cannot be debugged directly at runtime and does not have a user interface, interfaceless output of debugging information should be considered during development to facilitate debugging and troubleshooting. ii. To create a Port class that meets specific needs and use the Transceiver Kernel with a unified running and processing mechanism, the Ports in the Transceiver Shell are required to have unified processing rules. Some Ports in the Shell are existing component classes in the Delphi development environment (such as TCP, FTP, etc.), and some are not (such as MSMQ, File, etc.). At this time, you need to create a class that can meet your needs. For example: type//Since there is no user interface, it is inherited from TComponent instead of TControl TFilePort=class(TComponent) private FilePath:string;//Get or save the folder location of the file Prefix:string;//File prefix suffix:string; //File suffix end; After establishing the TFilePort class, Transceiver Kernel can use a unified class processing method to reference and manage objects to achieve the purpose of accessing specific files from the folder specified by FilePath. If used as Source, files that meet the conditions will be obtained from a specific folder. If used as Target, the data obtained from the corresponding Source will be written to the specified file (in fact The actual parameters of each Port object come from the definition of the Port table in the system configuration library). Another example: type TCOMPort=class(TComponent) private ComFace:string;//COM interface for obtaining or submitting dataend; TCOMPort will be used to obtain data from or submit data to the specified COM component interface. Carry out subsequent processing. In Delphi, the OleVariant class is one of the ways to implement COM component calls. The necessity of using the TCOMPort class is that Transceiver will instantiate the COM interface defined by TCOMPort into an OleVariant object only when necessary data access is required, and the object will be released after use. , This can reduce the load pressure on Transceiver and COM server. The same considerations apply to other similar components. The class example provided by the author here is just a model, and appropriate methods and events should be added when necessary. The classes implemented by the author during development include: TCOMPort, TMSMQPort, TDBPort, TFilePort, etc.iii. Support for multiple channels - an array of objects declaring Ports. Transceiver regards a communication process as a data flow process from source to target. Such a process is a Channel in Transceiver, and this Channel is composed of at least two It is composed of two ports (one for Source and one for Target), so if you want to define an indefinite number of Channels that can be freely combined with Source and Target, you must declare them separately for Source and Target. Arrays of objects of various Port classes (and establish corresponding relationships for them, as you will see later). For example: private { Private declarations }TCPSource:array of TServerSocket;//Object array for TCP Source TCPTarget:array of TClientSocket;//Object array for TCP Target MailSource:array of TIdPOP3; //For Mail Source Object array MailTarget:array of TIdSMTP; //Object array for Mail Target fileSource:array of TFilePort; //Object array for File Source fileTarget:array of TFilePort; //Object array for File Target comSource:array of TCOMPort; //Object array for COM Source comTarget:array of TCOMPort; //For COM Target object array Note: Since the Port operation rules for Source and Target of the same type are completely different, they are regarded as completely different objects with no direct relationship in the Transceiver concept. Therefore, for the same type of Port, object arrays are also created separately according to Source and Target. iv. Instantiate the object array at runtime. The number of elements in each object array is managed by Port Builder at runtime. If the user defines some ports of a certain type through the Transceiver Console, the Port Builder will instantiate them according to their number and respective parameters. The array of objects. Otherwise, the object array will not be instantiated. In the Source type Port object, the Name property is set to the form of 'Receive'+Port ID. In subsequent data reception triggers, this will help the Data Dispatcher locate the object and uniformly schedule different types of Port objects. The Tag attribute is used to provide the Channel Controller with the target ID information of the Channel in which it is located. The following is the instantiation part of the comSource object array in Port Builder begin //Create COM/ Receive Port itmp:=high(comSource)+1;// Get the current maximum number of comSource, itmp is the integer variable SetLength(comSource,itmp +1); // Add a comSource array member comSource [itmp]:=TCOMPort.Create(self); // Instantiate the member comSource[itmp].Name:= 'Receive'+inttostr(isource); //Set the Name attribute to 'Receive'+Port ID, and isource is the current PortID of the integer type comSource [itmp].Tag:= itarget; //Set it to the target ID of the Channel where it is located NullTest :=rece.Fields['Address'].value;//Get the value of system configuration COMFace, NullTest is a Variant variable if (NullTest <>null) and (trim(NullTest)<>'') then begincomSource [itmp].ComFace:=NullTest; //Assign valid values to ComFaceNullTest:=rece.Fields['interval'].value; //Get the trigger time interval for COM objects to obtain data in the system configuration SetTimer(application.handle,isource,NullTest*60000,nil); //Establish a trigger clock for the current Port for regularly collecting data, isource is Port IDendelsecomSource [itmp].Tag:=-1;//Initialization failed, marked as invalid Port end; comSource is the Source class Port used to call the interface defined in ComFace and obtain data after a certain time interval, corresponding to comTarget The implementation is similar, except that submitting data to ComFace of comTarget is a real-time process, so there is no need to use the trigger interval, and the two statements for establishing the clock can be omitted. The creation and initialization of other types of Port objects are similar. For example, another MailTarget implementation fragment: begin //Create SMTP/Send Port itmp:=high(MailTarget)+1; SetLength(MailTarget,itmp+1); MailTarget[itmp]:=TIdSMTP.Create(self); MailTarget[ itmp].Name:='send'+ inttostr(itarget); MailTarget[itmp].Tag:=3;//Set as Target Port type identification NullTest:=rece.Fields['Address'].value; //Mail server address if (NullTest <>null) and (trim(NullTest) <>'') then MailTarget[itmp].Host :=NullTest else bValid:=false; NullTest:=rece.Fields['Port'].value; //Mail server port if NullTest <>null then(if NullTest<>0 then MailTarget[itmp].Port :=NullTest)else bValid:=false; NullTest: =rece.Fields['user'].value;//Login user name if NullTest <>null thenMailTarget[itmp].UserId :=NullTest else bValid:=false; NullTest:=rece.Fields['password'].value;//Login password………………………end;Maybe you will have something like this I have some doubts. A large number of Transceiver Shell communication components are created by Port Builder at runtime. Will the performance of Transceiver Service be high? In fact, the mission of the Port Builder is completed once when the ServiceCreate event occurs. The number of Shell Ports will only affect the initialization speed of the Transceiver Service. The communication speed of the Shell Port and the overall performance of the Transceiver Service will not be affected. Of course, the system resources It might take up a little more. v. Dynamic allocation and processing of events Among several communication ports supported by Transceiver Shell, use TServerSocket (you may prefer to use Indy's communication component, but this does not violate the design idea of Transceiver Service, it is just a modification at the Shell level (or just added), the TCPSource implemented is a more distinctive one, because TServerSocket, as a Source Port, is different from objects such as COM or POP3 that need to be triggered regularly. It is in Transceiver The Service is always in a listening state after it is started. It is a component that generates corresponding events when a ClientSocket connects and sends data. The following is the instantiation fragment of TCPSource: begin //Create TCP/Receive Port itmp:=high(TCPSource)+1;SetLength(TCPSource,itmp+1); TCPSource [itmp]:=TServerSocket.Create(self); TCPSource [ itmp].OnClientRead:=TCPServersClientRead;//Assign the processing process of OnClientRead event to TCPServersClientRead TCPSource [itmp].OnClientError:=TCPServerClientError;//Assign the processing process of the OnClientError event to TCPServerClientErrorTCPSource [itmp].Name:= 'Receive'+inttostr(isource); //Set the Name property to 'Receive'+Port ID TCPSource [itmp ].Tag:=itarget; //Set the target IDTCPSource of its Channel [itmp].Socket.Data:=@ TCPSource [itmp].Tag;//Attach the target ID of this Port object to the Socket object as pointer data…………………………end; Come back and look at the processing of our comSource. During instantiation, we It establishes a trigger clock, but how to handle the events when the clock triggers? In the same way, it is also the dynamic allocation of event processing. The processing definition of comSource's clock can be added to the ServiceCreate event processing: application.OnMessage:=Timer; to implement the overloading of message processing. When a message from Application is generated, the Timer will be triggered. In the Timer event, we filter the processing With the WM_TIMER message triggered by the clock, you can call the data acquisition method of a specific Source Port according to the Port ID and type: Procedure TCarrier.Timer(var Msg: TMsg; var Handled: Boolean);var stmp:string; Obj:TComponent;begin if Msg.message =WM_TIMER then//Processing the clock message begin//Find the object that defines this message according to the Port ID that triggered the messageObj:=FindComponent('Receive'+inttostr (Msg.WParam)); if obj=nil then exit;//Exit processing if not found stmp:=obj.ClassName;//Reflect to obtain the type information of this Port object if stmp='TIdPOP3' then GetPOP3(TIdPOP3(Obj)); if stmp='TIdFTP' then GetFTP(TIdFTP(obj)); if stmp='TFilePort' then GetFile(TFilePort(Obj)); if stmp='TCOMPort' then GetCOM(TCOMPort(Obj));//Call the data acquisition process of COMSource…………………… end;end; vi. Get data The following is the data acquisition processing procedure of COMSource TCarrier.GetCOM(COMObj: TCOMPort);var stmp:string; COMInterface:OleVariant;begin try//Create a COM component object based on the value of ComFace COMInterface:=CreateOleObject (COMObj.ComFace); stmp:=COMInterface.GetData; //Call the agreed interface method to get data while stmp<>#0 do // #0 is the agreed data extraction end flag begin DataArrive(stmp, COMObj.Tag); // handed over to the data Dispatcher for unified processing, COMObj.Tag is the Target Port ID of the Channel where the object is located stmp:=COMInterface.GetData; end ; COMInterface:= Unassigned; except COMInterface:= Unassigned; end;end;// Complete the data extraction operation and release the component object until the next trigger call. The following is the data acquisition processing of TCPSource: procedure TCarrier.TCPServersClientRead(Sender: TObject; Socket:TCustomWinSocket);beginDataArrive(socket.ReceiveText,integer(TServerWinSocket(sender).data ^));//Left to data Dispatcher for unified processing, The second parameter is the Target Port ID pointer value attached to the Socket object sender, end; different types of Source Port objects receive data in different ways, but ultimately the received data is handed over to the data Dispatcher. Unified processing. From an implementation level, every time a data receiving object is added and its data reception is implemented, a new Source Port is implemented for Transceiver Shell. Note: The author here only implements receiving text data. Users may need to receive memory objects, data streams or binary data, and just make slight changes to the receiving code. vii. Data Scheduling The data scheduling of the Transceiver Service is completed by the data Dispatcher logical unit. The main task of the Data Dispatcher is to uniformly manage and control the data received from different Source Ports, and to work in conjunction with the Channel Controller. According to the Channel Define data distribution to different Target Ports, monitor whether the sending results are successful, and decide whether the data needs to be submitted to the Queue Manager and Log Recorder for buffering and log processing based on the sending results and the settings of the system configuration library. Next, look at the DataArrive method of Source Port submitting data: procedure TCarrier.DataArrive(sData:String;PortID:Integer);var dTime:Datetime; iLogID:integer; bSendSeccess:Boolean;begin if sData='' then exit;// If the data is empty, jump out iLogID:=-1; dTime:= now; //Receive time if sData[length(sdata)]=#0 then sdata:=copy(sdata,1,length(sdata)-1);//String format for C language compatibility bSendSeccess:=DataSend(sdata,PortID);//Call Data Dispatcher to send dispatch method, PortID is Target Port IDif (TSCfg.LogOnlyError=false) or (bSendSeccess=false) theniLogID:=writeLog(dTime, now,sData, PortID, bSendSeccess);//Record logs according to the log processing rules and sending results in the system configuration information if (TSCfg.Queueing=True) and (bSendSeccess=false) then PutQueue(dTime, now,sData, PortID, bSendSeccess, iLogID); / /Determine the Queue processing end based on the Queue configuration definition in the package system configuration information; the above is Data Dispatcher's DataArrive method, in which Queue processing is determined according to system configuration information and sending status, can also be adjusted to mandatory queuing processing. The following is the DataSend method of Data Dispatcher, which is used to distribute and process data according to the Target Port type: Function TCarrier.DataSend(sData:String;PortID:Integer):boolean;var Obj:TComponent;begin DataSend:=false;Obj:=FindComponent ('Send'+inttostr(PortID)); //Find the object based on the Port ID if (obj=nil) or (obj.Tag =-1) then exit;//The object does not exist or has been marked as invalid due to initialization failure. Port case obj.Tag of 1:DataSend:=PutTCP(TClientSocket(obj),sdata); 3:DataSend:=PutSMTP(TIdSMTP(obj), sdata); 5:DataSend:=PutFTP(TIdFTP(obj),sdata); 7:DataSend:=PutHTTP(TIdHTTP(obj),sdata); 9:DataSend:=PutFile(TFilePort(obj),sdata); 11:DataSend:=PutMSMQ(TMSMQPort (obj),sdata); 13:DataSend:= PutDB(TDBPort(obj),sdata); 15:DataSend:=PutCOM(TCOMPort (obj),sdata); …………… …………… end;end; It is worth noting that if an object array is not used, but there is only one instance of each type of Port If so, a better way to handle data distribution would be to use a callback function, but in the current case, that would lead to not knowing which member of the object array should handle the data. In addition, the current processing method does not completely separate Transceiver Kernel and Transceiver Shell, and we should seek a more abstract and independent processing method. viii. Data sending The following is the sending function of TCP TCarrier.PutTCP(TCPOBJ:TClientSocket;sdata:string):Boolean;var itime:integer;begin PutTCP:=false; try TCPOBJ.Close; TCPOBJ.Open; itime:=gettickcount; //Start time repeat application.ProcessMessages; until (TCPOBJ.Active=true) or (gettickcount-itime>5000); //Jump out of the loop if the connection is successful or the 5-second timeout occurs if TCPOBJ.Active then begin TCPOBJ.Socket.SendText(sdata); PutTCP:=true;//The return value is only when the data is sent successfully. Trueend;TCPOBJ.Close; ExceptTCPOBJ.Close; end; end;The following is the sending function of COM TCarrier.PutCOM(COMOBJ:TCOMPort;sdata:string):Boolean;var Com:OleVariant;begin PutCOM:=false; try Com:=CreateOleObject(COMOBJ.ComFace);//Create a predefined interface PutCOM:=Com.PutData (sdata);//Call the predefined method Com:= Unassigned; exceptCom:= Unassigned; end; end; Other types of Port sending are similar and will not be repeated here. So far, the basic processing of Source and Target has been completed. A basic communication function has been established. After free matching of different types of Source and Target, completely different communication functions can be realized. By creating multiple Channels, you can centrally implement communication processing for multiple different functions. ix. Queue processing: After the data is sent in the DataArrive method above, Data Dispatcher will call the writeLog for data logging and the PutQueue method for queue processing. The functions of the two are similar. They both process the data information according to the system parameters. Storage is not the focus of this article. The Retry processing of the queue is similar to the principle of distributing processing by Port type in the Timer event. It relies on the trigger of the Queue Timer to read the buffered data from the database and call DataSend again according to the Target Port ID to retry sending the data. , if the transmission is successful, the transaction of this data transmission is completed, otherwise it will re-enter the queue and wait for the next trigger time to retry until the transmission is successful or the set maximum number of retries is reached. three, Summary of Development Experience Since the focus of this article is to explain the core ideas and design concepts of Transceiver, it simplifies and weakens the multi-threading processing, object pooling and transaction support that Transceiver should consider as a background service, and the more complex and powerful Source and Target Groups. Management and Cha nnel integration, the ability to send and receive memory objects, data streams, binary data, the reading of system configuration information and the implementation of its encapsulation classes, the security of the system and data, etc. I hope readers can provide some insights and understand the design ideas of Transceiver. Inspire sparks of inspiration in actual development work and create more outstanding and powerful software. Author: Firebird [email protected] Use Delphi to establish communication and data exchange servers—Transceiver technical analysis (Part 1) Use Delphi to establish communication and data exchange servers—Transceiver technical analysis (Part 2) Implement collection classes through C# Overview of .NET Collections and related Old technical things: program shortcuts/program deletions/EXE self-deleting DIY old things: childhood programming algorithm experience notes