apache flink's machine learning library is called flinkml. since usage of machine learning has been increasing exponentially over the last 5 years, flink community decided to add this machine learning apo also in its ecosystem. the list of contributors and algorithms are increasing in flinkml. this api is not a part of binary distribution yet.

here is an example of linear regression using flinkml −

// labeledvector is a feature vector with a label (class or real value)
val trainingdata: dataset[labeledvector] = ...
val testingdata: dataset[vector] = ...

// alternatively, a splitter is used to break up a dataset into training and testing data.
val dataset: dataset[labeledvector] = ...
val traintestdata: dataset[traintestdataset] = splitter.traintestsplit(dataset)
val trainingdata: dataset[labeledvector] = traintestdata.training
val testingdata: dataset[vector] = traintestdata.testing.map(lv => lv.vector)
val mlr = multiplelinearregression()

.setstepsize(1.0)
.setiterations(100)
.setconvergencethreshold(0.001)
mlr.fit(trainingdata)

// the fitted model can now be used to make predictions
val predictions: dataset[labeledvector] = mlr.predict(testingdata)

inside flink-1.7.1/examples/batch/ path, you will find kmeans.jar file. let us run this sample flinkml example.

this example program is run using the default point and the centroid data set.

./bin/flink run examples/batch/kmeans.jar --output print

Save or share this article