converts the input array of strings into an array of n grams

Solutions on MaxInterview for converts the input array of strings into an array of n grams by the best coders in the world

showing results for - "converts the input array of strings into an array of n grams"
Emmy
06 Apr 2017
1# converts the input array of strings into an array of n-grams
2
3df = spark.createDataFrame([Row(inputTokens=[
4  "a", "b", "c", "d", "e"])])
5ngram = NGram(n=2, inputCol="inputTokens", outputCol="nGrams")
6ngram.transform(df).head()
7# Row(inputTokens=[u'a', u'b', u'c', u'd', u'e'], nGrams=[u'ab', u'b c', u'c d', u'd e'])
8
9# Change n-gram Length
10ngram.setParams(n=4).transform(df).head()
11# Row(inputTokens=[u'a', u'b', u'c', u'd', u'd', u'e'], nGrams=[u'a b c d', u'b c d e'])
12
13# Temporarily modify output column.
14ngram.transform(df, {ngram.outputCol: "output"}).head()
15# Row(inputTokens=[u'a', u'b', u'c', u'd', u'e'], output=[u'a b c d', u'b c d e'])
16ngram.transform(df).head()
17# Row(inputTokens=[u'a', u'b', u'c', u'd', u'e'], nGrams=[u'a b c d', u'b c d e'])
18
19# Must use keyword arguments to specify params.
20ngram.setParams("text")
21# Traceback (most recent call last):
22#	...
23# TypeError: Method setParams forces keyword arguments.
24ngramPath = temp_path + "/ngram"
25ngram.save(ngramPath)
26loadedNGram = NGram.load(ngramPath)
27loadedNGram.getN() == ngram.getN()
28# True